memleak-bpfcc(8)

Usage

memleak [-h] [-p PID] [-t] [-a] [-o OLDER] [-c COMMAND] [--combined-only] [--wa-missing-free] [-s SAMPLE_RATE] [-T TOP] [-z MIN_SIZE] [-Z MAX_SIZE] [-O OBJ] [INTERVAL] [COUNT]

Overview

memleak traces and matches memory allocation and deallocation requests, and collects call stacks for each allocation. memleak can then print a summary of which call stacks performed allocations that weren't subsequently freed.

When tracing a specific process, memleak instruments a list of allocation functions from libc, specifically: malloc, calloc, realloc, posix_memalign, valloc, memalign, pvalloc, aligned_alloc, and free. When tracing all processes, memleak instruments kmalloc/kfree, kmem_cache_alloc/kmem_cache_free, and also page allocations made by get_free_pages/free_pages.

memleak may introduce significant overhead when tracing processes that allocate and free many blocks very quickly. See the OVERHEAD section below.

This tool only works on Linux 4.6+. Stack traces are obtained using the new BPF_STACK_TRACE` APIs. For kernels older than 4.6, see the version under tools/old. Kernel memory allocations are intercepted through tracepoints, which are available on Linux 4.7+.

Requirements

CONFIG_BPF and bcc.

Options

-h: Print usage message.

-p PID: Trace this process ID only (filtered in-kernel). This traces libc allocator.

-t: Print a trace of all allocation and free requests and results.

-a: Print a list of allocations that weren't freed (and their sizes) in addition to their call stacks.

-o OLDER: Print only allocations older than OLDER milliseconds. Useful to remove false positives. The default value is 500 milliseconds.

-c COMMAND: Run the specified command and trace its allocations only. This traces libc allocator.

--combined-only: Use statistics precalculated in kernel space. Amount of data to be pulled from kernel significantly decreases, at the cost of losing capabilities of time-based false positives filtering (-o).

--wa-missing-free: Make up the action of free to alleviate misjudgments when free is missing.

-s SAMPLE_RATE: Record roughly every SAMPLE_RATE-th allocation to reduce overhead.

-t TOP: Print only the top TOP stacks (sorted by size). The default value is 10.

-z MIN_SIZE: Capture only allocations that are larger than or equal to MIN_SIZE bytes.

-Z MAX_SIZE: Capture only allocations that are smaller than or equal to MAX_SIZE bytes.

-O OBJ: Attach to allocation functions in specified object instead of resolving libc. Ignored when kernel allocations are profiled.

INTERVAL: Print a summary of outstanding allocations and their call stacks every INTERVAL seconds. The default interval is 5 seconds.

COUNT: Print the outstanding allocations summary COUNT times and then exit.

Examples

`Print outstanding kernel allocation stacks every 3 seconds:`	# memleak 3
`Print user outstanding allocation stacks and allocation details for the process 1005:`	# memleak -p 1005 -a
`Sample roughly every 5th allocation (~20%) of the call stacks and print the top 5`	stacks 10 times before quitting. # memleak -s 5 --top=5 10
`Run ./allocs and print outstanding allocation stacks for that process:`	# memleak -c ./allocs
`Capture only allocations between 16 and 32 bytes in size:`	# memleak -z 16 -Z 32

Overhead

memleak can have significant overhead if the target process or kernel performs allocations at a very high rate. Pathological cases may exhibit up to 100x degradation in running time. Most of the time, however, memleak shouldn't cause a significant slowdown. You can use the -s switch to reduce the overhead further by capturing only every N-th allocation. The -z and -Z switches can also reduce overhead by capturing only allocations of specific sizes.

Additionally, option --combined-only saves processing time by reusing already calculated allocation statistics from kernel. It's faster, but lacks information about particular allocations.

Also, option --wa-missing-free makes memleak more accuracy in the complicated environment.

To determine the rate at which your application is calling malloc/free, or the rate at which your kernel is calling kmalloc/kfree, place a probe with perf and collect statistics. For example, to determine how many calls to __kmalloc are placed in a typical period of 10 seconds:

# perf probe '__kmalloc'

# perf stat -a -e 'probe:__kmalloc' -- sleep 10

Source

This is from bcc.

: https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _examples.txt file containing example usage, output, and commentary for this tool.

OS

Linux

Stability

Unstable - in development.

Releases

4 releases since Debian 10 (Buster); the text changed 1 time.

10
11–13

--wa-missing-free since Debian 11 (Bullseye).