stxxl_tool (1)

stxxl_tool <subtool> ...

The stxxl_tool is an umbrella program that houses several sub-tools for STXXL relating to (e.g.) scratch file manipulation and benchmarking. The behaviour of each sub-tool is given below.

Size arguments will accept a string like "343KB" or "44 GiB" as their input.

Subtools

info

Print out information about the build system and which optional modules where compiled into STXXL.

create_files <filesize> <filename ...>

Pre-create large files to keep file system allocation time out to measurements.

benchmark_disks [options] <size> [r|w] [alloc]

Benchmark the disks configured by the standard .stxxl disk configuration files mechanism. Blocks of 8 MiB are written and/or read in sequence using the block manager. The batch size describes how many blocks are written/read in one batch. The are taken from block_manager using given the specified allocation strategy. If size == 0, then writing/reading operation are done until an error occurs.

Parameters:

  size    Amount of data to write/read from disks (e.g. 10GiB)
  r|w     Only read or write blocks (default: both write and read)
  alloc   Block allocation strategy: RC, SR, FR, striping. (default: RC)

Options:

  -b, --batch  Number of blocks written/read in one batch (default: D * 8MiB)

benchmark_files [options] <length> <filename ...>

Open a file using one of STXXL's file abstractions and perform write/read/verify tests on the file. Block sizes and batch size can be adjusted via command line. If length == 0 , then operation will continue till end of space (please ignore the write error). Memory consumption: block_size * batch_size * num_files

Parameters:

  length    Length to write in file.
  filename  File path to run benchmark on.

Options:

  -o, --offset      Starting offset to write in file.
  --no-direct       open files without O_DIRECT
  --sync            open files with O_SYNC|O_DSYNC|O_RSYNC
  --resize          resize the file size after opening, needed e.g. for
                    creating mmap files
  --block_size      block size for operations (default 8 MiB)
  --batch_size      increase (default 1) to submit several I/Os at once and
                    report average rate
  -f, --file-type   Method to open file (syscall|mmap|wincall|boostfd|...)
                    default: syscall
  -p, --operations  [w]rite pattern, [r]ead without verification, read and
                    [v]erify pattern (default: 'wv')
  --pattern         32-bit pattern to write (default: block index)

Benchmark different file access methods, e.g. syscall or mmap_files.

benchmark_sort [options] <size>

Run benchmark tests of different sorting methods in STXXL. size is the amount of data to sort, in GiB

Valid options are: -M, --ram : amount of ram to use when sorting, default 256 MiB

benchmark_disks_random <span> [block_size] [size] [i|r|w] [alloc]

Benchmark _random_ block access on the disks configured by the standard .stxxl disk configuration files mechanism. Available block sizes are power of two from 4 KiB to 128 MiB. A set of three operations can be performed: sequential initialization, random reading and random writing.

Parameters:

  span        Span of external memory to write/read to (e.g. 10GiB).
  block_size  Size of blocks to randomly write/read (default: 8MiB).
  size        Amount of data to operate on (e.g. 2GiB), default: whole span.
  i|r|w       Operations: [i]nitialize, [r]ead, and/or [w]rite (default: all).
  alloc       Block allocation strategy: RC, SR, FR, striping (default: RC).

benchmark_pqueue [options] [size]

Benchmark the priority queue implementation using a sequence of operations. The PQ contains pairs of 32- or 64-bit integers, or a 24 byte struct. The operation sequence is either a simple fill/delete cycle or fill/intermixed inserts/deletes. Because the memory parameters of the PQ must be set a compile-time, the benchmark provides only three PQ sizes: for 256 MiB, 1 GiB and 8 GiB of RAM, with the maximum number of items set accordingly.

Parameters:

  size    Amount of data to insert (e.g. 1GiB)

Options:

  -t, --type   Value type of tested priority queue:
         1 = pair of uint32,
         2 = pair of uint64 (default),
         3 = 24 byte struct
         0 = all of the above
  -p, --pq     Priority queue configuration to test:
         1 = small (256 MiB RAM, 4 GiB elements)
         2 = medium (1 GiB RAM, 16 GiB elements) (default)
         3 = big (8 GiB RAM, 64 GiB elements)
         0 = all of the above
  -o, --opseq  Operation sequence to perform:
         1 = insert all, delete all (default)
         2 = insert all, intermixed insert/delete
         0 = all of the above

mlock <size>

Allocate some memory and mlock() it to consume physical memory. Needs to run as root to block more than 64 KiB in default settings.

mallinfo

Show mallinfo statistics.

stxxl is written by many authors, and can be found in the AUTHORS section of the documentation package, or on the homepage

libstxxl Project Home : http://stxxl.sourceforge.net

This manual page was written by D Haley <mycae gmx com>, for the Debian project (and may be used by others).