parsort1

Usage

Overview

parsort uses GNU sort to sort in parallel. It works just like sort but faster on inputs with more than 1 M lines, if you have a multicore machine.

Hopefully these ideas will make it into GNU sort in the future.

Options

Same as sort. Except:

--parallel=N

Change the number of sorts run concurrently to N. N will be increased to number of files if parsort is given more than N files.

Examples

Sort files:

  parsort *.txt > sorted.txt

Sort stdin (standard input) numerically:

  cat numbers | parsort -n > sorted.txt

Performance

parsort is faster on files than on stdin (standard input), because different parts of a file can be read in parallel.

On a 48 core machine you should see a speedup of 3x over sort.

Dependencies

parsort uses sort, bash, and parallel.