trafgen8

Usage

Overview

trafgen is a fast, zero-copy network traffic generator for debugging, performance evaluation, and fuzz-testing. trafgen utilizes the packet(7) socket interface of Linux which postpones complete control over packet data and packet headers into the user space. It has a powerful packet configuration language, which is rather low-level and not limited to particular protocols. Thus, trafgen can be used for many purposes. Its only limitation is that it cannot mimic full streams resp. sessions. However, it is very useful for various kinds of load testing in order to analyze and subsequently improve systems behaviour under DoS attack scenarios, for instance.

trafgen is Linux specific, meaning there is no support for other operating systems, same as netsniff-ng(8), thus we can keep the code footprint quite minimal and to the point. trafgen makes use of packet(7) socket's TX_RING interface of the Linux kernel, which is a mmap(2)'ed ring buffer shared between user and kernel space.

By default, trafgen starts as many processes as available CPUs, pins each of them to their respective CPU and sets up the ring buffer each in their own process space after having compiled a list of packets to transmit. Thus, this is likely the fastest one can get out of the box in terms of transmission performance from user space, without having to load unsupported or non-mainline third-party kernel modules. On Gigabit Ethernet, trafgen has a comparable performance to pktgen, the built-in Linux kernel traffic generator, except that trafgen is more flexible in terms of packet configuration possibilities. On 10-Gigabit-per-second Ethernet, trafgen might be slower than pktgen due to the user/kernel space overhead but still has a fairly high performance for out of the box kernels.

trafgen has the potential to do fuzz testing, meaning a packet configuration can be built with random numbers on all or certain packet offsets that are freshly generated each time a packet is sent out. With a built-in IPv4 ping, trafgen can send out an ICMP probe after each packet injection to the remote host in order to test if it is still responsive/alive. Assuming there is no answer from the remote host after a certain threshold of probes, the machine is considered dead and the last sent packet is printed together with the random seed that was used by trafgen. You might not really get lucky fuzz-testing the Linux kernel, but presumably there are buggy closed-source embedded systems or network driver's firmware files that are prone to bugs, where trafgen could help in finding them.

trafgen's configuration language is quite powerful, also due to the fact, that it supports C preprocessor macros. A stddef.h is being shipped with trafgen for this purpose, so that well known defines from Linux kernel or network programming can be reused. After a configuration file has passed the C preprocessor stage, it is processed by the trafgen packet compiler. The language itself supports a couple of features that are useful when assembling packets, such as built-in runtime checksum support for IP, UDP and TCP. Also it has an expression evaluator where arithmetic (basic operations, bit operations, bit shifting, ...) on constant expressions is being reduced to a single constant on compile time. Other features are ''fill'' macros, where a packet can be filled with n bytes by a constant, a compile-time random number or run-time random number (as mentioned with fuzz testing). Also, netsniff-ng(8) is able to convert a pcap file into a trafgen configuration file, thus such a configuration can be further tweaked for a given scenario.

Options

-i <cfg|pcap|->, -c <cfg|->, --in <cfg|pcap|->, --conf <cfg|->

Defines the input configuration file that can either be passed as a normal plain text file or via stdin (''-''). Note that currently, if a configuration is passed through stdin, only 1 CPU will be used. It is also possible to specify PCAP file with .pcap extension via -i/--in option, by default packets will be sent at rate considering timestamp from PCAP file which might be reset via the -b or -t option.

-o <dev|.pcap|.cfg>, -d <dev|.pcap|.cfg>, --out <dev|.pcap|.cfg>, --dev <dev|.pcap|.cfg>

Defines the outgoing networking device such as eth0, wlan0 and others or a *.pcap or *.cfg file. Pcap and configuration files are identified by extension.

-p, --cpp

Pass the packet configuration to the C preprocessor before reading it into trafgen. This allows #define and #include directives (e.g. to include definitions from system headers) to be used in the trafgen configuration file.

-D <name>=<definition>, --define <name>=<definition>

Add macro definition for the C preprocessor to use it within trafgen file. This option is used in combination with the -p/--cpp option.

-J, --jumbo-support

By default trafgen's ring buffer frames are of a fixed size of 2048 bytes. This means that if you're expecting jumbo frames or even super jumbo frames to pass your line, then you will need to enable support for that with the help of this option. However, this has the disadvantage of a performance regression and a bigger memory footprint for the ring buffer.

-R, --rfraw

In case the output networking device is a wireless device, it is possible with trafgen to turn this into monitor mode and create a mon<X> device that trafgen will be transmitting on instead of wlan<X>, for instance. This enables trafgen to inject raw 802.11 frames. In case if the output is a pcap file the link type is set to 127 (ieee80211 radio tap).

-s <ipv4>, --smoke-test <ipv4>

In case this option is enabled, trafgen will perform a smoke test. In other words, it will probe the remote end, specified by an <ipv4> address, that is being ''attacked'' with trafgen network traffic, if it is still alive and responsive. That means, after each transmitted packet that has been configured, trafgen sends out ICMP echo requests and waits for an answer before it continues. In case the remote end stays unresponsive, trafgen assumes that the machine has crashed and will print out the content of the last packet as a trafgen packet configuration and the random seed that has been used in order to reproduce a possible bug. This might be useful when testing proprietary embedded devices. It is recommended to have a direct link between the host running trafgen and the host being attacked by trafgen.

-n <0|uint>, --num <0|uint>

Process a number of packets and then exit. If the number of packets is 0, then this is equivalent to infinite packets resp. processing until interrupted. Otherwise, a number given as an unsigned integer will limit processing.

-r, --rand

Randomize the packet selection of the configuration file. By default, if more than one packet is defined in a packet configuration, packets are scheduled for transmission in a round robin fashion. With this option, they are selected randomly instread.

-P <uint>[-<uint>], --cpus <uint>[-<uint>]

Specify the number of processes trafgen shall fork(2) off or list exact CPUs to use. By default trafgen will start as many processes as CPUs that are online and pin them to each, respectively. A single integer within interval [1,CPUs] overrides number of processes, which will be spawned starting from the first CPU. A pair of integers within interval [0,CPUs-1], and separated using ''-'' represents an interval of CPUs, which will be used to spawn worker processes.

-t <time>, --gap <time>

Specify a static inter-packet timegap in seconds, milliseconds, microseconds, or nanoseconds: ''<num>s/ms/us/ns''. If no postfix is given default to microseconds. If this option is given, then instead of packet(7)'s TX_RING interface, trafgen will use sendto(2) I/O for network packets, even if the <time> argument is 0. This option is useful for a couple of reasons:

1) comparison between sendto(2) and TX_RING performance,
2) low-traffic packet probing for a given interval,
3) ping-like debugging with specific payload patterns.

Furthermore, the TX_RING interface does not cope with interpacket gaps.

-b <rate>, --rate <rate>

Specify the packet send rate <num>pps/kpps/Mpps/B/kB/MB/GB/kbit/Mbit/Gbit/KiB/MiB/GiB units. Like with the -t/--gap option, the packets are sent in slow mode.

-S <size>, --ring-size <size>

Manually define the TX_RING resp. TX_RING size in ''<num>KiB/MiB/GiB''. By default the size is being determined based on the network connectivity rate.

-E <uint>, --seed <uint>

Manually set the seed for pseudo random number generator (PRNG) in trafgen. By default, a random seed from /dev/urandom is used to feed glibc's PRNG. If that fails, it falls back to the unix timestamp. It can be useful to set the seed manually in order to be able to reproduce a trafgen session, e.g. after fuzz testing.

-u <uid>, --user <uid> resp. -g <gid>, --group <gid>

After ring setup, drop privileges to a non-root user/group combination.

-H, --prio-high

Set this process as a high priority process in order to achieve a higher scheduling rate resp. CPU time. This is however not the default setting, since it could lead to starvation of other processes, for example low priority kernel threads.

-A, --no-sock-mem

Do not change systems default socket memory setting during testrun. Default is to boost socket buffer memory during the test to:

/proc/sys/net/core/rmem_default:4194304
/proc/sys/net/core/wmem_default:4194304
/proc/sys/net/core/rmem_max:104857600
/proc/sys/net/core/wmem_max:104857600

-Q, --notouch-irq

Do not reassign the NIC's IRQ CPU affinity settings.

-q, --qdisc-path

Since Linux 3.14, the kernel supports a socket option PACKET_QDISC_BYPASS, which trafgen enables by default. This options disables the qdisc bypass, and uses the normal send path through the kernel's qdisc (traffic control) layer, which can be usefully for testing the qdisc path.

-V, --verbose

Let trafgen be more talkative and let it print the parsed configuration and some ring buffer statistics.

-e, --example

Show a built-in packet configuration example. This might be a good starting point for an initial packet configuration scenario.

-C, --no-cpu-stats

Do not print CPU time statistics on exit.

-v, --version

Show version information and exit.

-h, --help

Show user help and exit.

Usage example

trafgen --dev eth0 --conf trafgen.cfg

This is the most simple and, probably, the most common use of trafgen. It will generate traffic defined in the configuration file ''trafgen.cfg'' and transmit this via the ''eth0'' networking device. All online CPUs are used.

trafgen --dev eth0 --conf trafgen.cfg --cpus 2-4

Instead of using all online CPUs, transmit traffic from CPUs 2, 3, and 4.

trafgen -e | trafgen -i - -o lo --cpp -n 1

This is an example where we send one packet of the built-in example through the loopback device. The example configuration is passed via stdin and also through the C preprocessor before trafgen's packet compiler will see it.

trafgen --dev eth0 --conf fuzzing.cfg --smoke-test 10.0.0.1

Read the ''fuzzing.cfg'' packet configuration file (which contains drnd() calls) and send out the generated packets to the ''eth0'' device. After each sent packet, ping probe the attacked host with address 10.0.0.1 to check if it's still alive. This also means, that we utilize 1 CPU only, and do not use the TX_RING, but sendto(2) packet I/O due to ''slow mode''.

trafgen --dev wlan0 --rfraw --conf beacon-test.txf -V --cpus 2

As an output device ''wlan0'' is used and put into monitoring mode, thus we are going to transmit raw 802.11 frames through the air. Use the ''beacon-test.txf'' configuration file, set trafgen into verbose mode and use only 2 CPUs starting from CPU 0.

trafgen --dev em1 --conf frag_dos.cfg --rand --gap 1000us

Use trafgen in sendto(2) mode instead of TX_RING mode and sleep after each sent packet a static timegap for 1000us. Generate packets from ''frag_dos.cfg'' and select next packets to send randomly instead of a round-robin fashion. The output device for packets is ''em1''.

trafgen --dev eth0 --conf icmp.cfg --rand --num 1400000 -k1000

Send only 1400000 packets using the ''icmp.cfg'' configuration file and then exit trafgen. Select packets randomly from that file for transmission and send them out via ''eth0''. Also, trigger the kernel every 1000us for batching the ring frames from user space (default is 10us).

trafgen --dev eth0 --conf tcp_syn.cfg -u `id -u bob` -g `id -g bob`

Send out packets generated from the configuration file ''tcp_syn.cfg'' via the ''eth0'' networking device. After setting up the ring for transmission, drop credentials to the non-root user/group bob/bob.

trafgen --dev eth0 '{ fill(0xff, 6), 0x00, 0x02, 0xb3, rnd(3), c16(0x0800), fill(0xca, 64) }' -n 1

Send out 1 invaid IPv4 packet built from command line to all hosts.

Note

trafgen can saturate a Gigabit Ethernet link without problems. As always, of course, this depends on your hardware as well. Not everywhere where it says Gigabit Ethernet on the box, will you reach almost physical line rate! Please also read the netsniff-ng(8) man page, section NOTE for further details about tuning your system e.g. with tuned(8).

If you intend to use trafgen on a 10-Gbit/s Ethernet NIC, make sure you are using a multiqueue tc(8) discipline, and make sure that the packets you generate with trafgen will have a good distribution among tx_hashes so that you'll actually make use of multiqueues.

For introducing bit errors, delays with random variation and more, there is no built-in option in trafgen. Rather, one should reuse existing methods for that which integrate nicely with trafgen, such as tc(8) with its different disciplines, i.e. netem.

For more complex packet configurations, it is recommended to use high-level scripting for generating trafgen packet configurations in a more automated way, i.e. also to create different traffic distributions that are common for industrial benchmarking:

Traffic model Distribution

IMIX 64:7, 570:4, 1518:1
Tolly 64:55, 78:5, 576:17, 1518:23
Cisco 64:7, 594:4, 1518:1
RPR Trimodal 64:60, 512:20, 1518:20
RPR Quadrimodal 64:50, 512:15, 1518:15, 9218:20

The low-level nature of trafgen makes trafgen rather protocol independent and therefore useful in many scenarios when stress testing is needed, for instance. However, if a traffic generator with higher level packet descriptions is desired, netsniff-ng's mausezahn(8) can be of good use as well.

For smoke/fuzz testing with trafgen, it is recommended to have a direct link between the host you want to analyze (''victim'' machine) and the host you run trafgen on (''attacker'' machine). If the ICMP reply from the victim fails, we assume that probably its kernel crashed, thus we print the last sent packet together with the seed and quit probing. It might be very unlikely to find such a ping-of-death on modern Linux systems. However, there might be a good chance to find it on some proprietary (e.g. embedded) systems or buggy driver firmwares that are in the wild. Also, fuzz testing can be done on raw 802.11 frames, of course. In case you find a ping-of-death, please mention that you were using trafgen in your commit message of the fix!

Legal

trafgen is licensed under the GNU GPL version 2.0.