nbdkit-log-filter1

Usage

Overview

"nbdkit-log-filter" is a filter that logs all transactions to a file or external script.

When used as the first filter, it can show the original client requests. As a later filter, it can show how earlier filters have modified the original request.

When using "logfile=FILE", logs are written to a log file with the format described in "LOG FILE FORMAT" below.

When using "logscript=SCRIPT", logs invoke the external script. See "LOG SCRIPT" below.

An alternative to this filter is simply to run nbdkit with the -f and -v flags which enable verbose debugging to stderr. This logs many aspects of nbdkit operation, but requires running nbdkit in the foreground. The log filter uses a more parsimonious and more easily parsable format and works when nbdkit runs in the background.

Parameters

"logfile" or "logscript" or both can be given. If neither then the filter is inactive.

logfile=FILE

The file where the log is written. See "LOG FILE FORMAT" below.

logscript=SCRIPT

(nbdkit ≥ 1.24)

Log lines invoke an external script. See "LOG SCRIPT" below.

logappend=true

logappend=false

(nbdkit ≥ 1.8)

This only affects "logfile". If "false" (the default), if the file already exists it will be truncated. If "true", the filter appends to the existing log file.

Examples

Serve the file disk.img, and log each client transaction in the file disk.log:

 nbdkit --filter=log file disk.img logfile=disk.log

Repeat the task, but with the cow (copy-on-write) filter to perform local caching of data served from the original plugin:

 nbdkit --filter=cow --filter=log file disk.img logfile=disk.log2

After running a client that performs the same operations under each of the two servers, you can compare disk.log and disk.log2 to see the impact of the caching.

Log file format

An example logging session of a client that requests an export list before performing a single successful read is:

 2020-08-06 02:07:23.080415 ListExports id=1 readonly=0 tls=0 ...
 2020-08-06 02:07:23.080502 ...ListExports id=1 exports=("") return=0
 2020-08-06 02:07:23.080712 connection=1 Connect export="" tls=0 size=0x400 minsize=0x1 prefsize=0x200 maxsize=0xffffffff write=1 flush=1 rotational=0 trim=1 zero=2 fua=2 extents=1 cache=2 fast_zero=1
 2020-08-06 02:07:23.080907 connection=1 Read id=1 offset=0x0 count=0x200 ...
 2020-08-06 02:07:23.080927 connection=1 ...Read id=1 return=0
 2020-08-06 02:07:23.081255 connection=1 Disconnect transactions=1

All lines start with a timestamp in "YYYY-MM-DD HH:MM:ZZ.MS" format.

For connected calls, "connection=N" is present to distinguish between clients.

The action follows. Currently the following actions are logged: ListExports, Ready, Fork, Preconnect, Connect, Read, Write, Zero, Trim, Extents, Cache, Flush and Disconnect.

Some actions are logged across two lines showing the call and return value. Because nbdkit handles requests in parallel different requests may be intermingled. Use the "id=N" field for correlation, it is unique per connection.

Strings and lists are shell-quoted.

Log script

If "logscript=SCRIPT" is given on the command line then log entries are passed to the external script.

The script is passed several shell variables:

$act

The action name, like "Read", "Write" etc.

$connection

The connection ID identifying the client, only for connected calls like "Read".

$error

For messages of type "LEAVE" which fail ("$return = -1"), this contains the errno as a string, for example "EIO".

$id

The transaction ID, used to correlate actions which are split into two messages "ENTER" and "LEAVE".

$return

For messages of type "LEAVE" this is the return code, usually 0 for success and -1 if there was an error.

$type

The message type: "ENTER", "LEAVE" or "PRINT".

other shell variables

Other parameters like "offset=N" are turned into shell variables $offset etc.

Note the return value of the script is ignored. Log scripts cannot modify or interrupt request processing.

 nbdkit -f --filter=log null 10M \
        logscript='echo $connection $type $id $act $offset >&2'

 PRINT Ready
 1 ENTER 1 Read 0x0
 1 ENTER 2 Write 0x200
 1 LEAVE 2 Write
 1 LEAVE 1 Read

 Ready thread_model=3
 connection=1 Read id=1 offset=0x0 count=0x200 ...
 connection=1 Write id=2 offset=0x200 count=0x200 ...
 connection=1 ...Write id=2
 connection=1 ...Read id=1

 nbdkit -f --filter=log memory 10M \
        logscript='
            if [ "$act" = "Read" -a "$type" = "ENTER" ]; then
                echo Client is reading $count bytes from $offset >&2
            fi
        '

Files

"logfile=FILE" parameter

This filter writes to the file specified by the "logfile=FILE" parameter.

$filterdir/nbdkit-log-filter.so

The filter.

Use "nbdkit --dump-config" to find the location of $filterdir.

Version

"nbdkit-log-filter" first appeared in nbdkit 1.4.