perf-stat-hist (8)

perf-stat-hist [-h] [-b buckets|-P power] [-m max] tracepoint variable [seconds]

This is a proof-of-concept showing in-kernel histograms using Linux perf_events (aka the "perf" command), on older kernels where perf_events does not have this native capability.

These histograms show the distribution of variable, allowing details including multiple modes and outliers to be studied.

This uses multiple counting tracepoints with different filters, one for each histogram bucket. While this is summarized in-kernel, the use of multiple tracepoints does add addiitonal overhead. Hopefully, in the future this this functionality will be provided in an efficient way from perf_events itself, at which point this tool can be deleted or rewritten.

Linux perf_events: add linux-tools-common, run "perf", then add any additional packages it requests. Also uses awk.

-h

Usage message.

-b buckets

Specify a list of bucket points for the histogram as a string (eg, "10 500 1000"). The histogram will include buckets for less-than the minimum, and greater-than-or-equal-to the maximum. If a single value is specified, two statistics only are gathered: for less-than and for greater-than-or-equal-to. The overhead is relative to the number of buckets, so only specifying a single value costs the lowest overhead.

-P power

Power for power-of histogram. By default, a power-of-4 histogram is created. This and the -b option are exclusive.

-m max

Max value for power-of histograms.

tracepoint

Tracepoint specification. Eg, syscalls:sys_enter_read.

variable

The tracepoint variable name to summarize. To see what are available, cat the format file under /sys/kernel/debug/tracing/events/*/*/format.

seconds

Number of seconds to trace. If not specified, this runs until Ctrl-C.

Trace read() syscalls until Ctrl-C, and show histogram of requested size:

# perf-stat-hist syscalls:sys_enter_read count

Trace read() syscall completions until Ctrl-C, and show histogram of successful returned size:

# perf-stat-hist syscalls:sys_exit_read ret

Trace read() return sizes for 10 seconds, showing histogram:

# perf-stat-hist syscalls:sys_exit_read ret 10

Trace network transmits until Ctrl-C, and show histogram of packet size:

# perf-stat-hist net:net_dev_xmit len

Trace read() return sizes, using a power-of-10 histogram:

perf-stat-hist -P 10 syscalls:sys_exit_read ret

Trace read() return sizes, using a power-of-2 histogram, and a max of 1024:

perf-stat-hist -P 2 -m 1024 syscalls:sys_exit_read ret

Trace read() return sizes, using the specified bucket points:

perf-stat-hist -b 10 50 100 5000 syscalls:sys_exit_read ret

Trace read() return sizes, and bifurcate statistics by the value 10:

perf-stat-hist -b 10 syscalls:sys_exit_read ret

Range

Range of the histogram bucket, in units of the variable specified.

Count

Number of occurrences (tracepoint events) of the variable in this range.

Distribution

ASCII histogram representation of the Count column.

While the counts are performed in-kernel, there is one tracepoint used per histogram bucket, so the overheads are higher than usual (relative to the number of buckets) than function counting using perf stat. The lowest overhead is when -b is used to specify one bucket only, bifurcating statistics.

This is from the perf-tools collection.

• https://github.com/brendangregg/perf-tools

Also look under the examples directory for a text file containing example usage, output, and commentary for this tool.

Linux

Unstable - in development.

Brendan Gregg

perf(1)