xtokid (1)

xtokid [OPTION]... [FILE]...

Print all tokens found in a source file.

-i, --include=LANGS

include languages in LANGS (default: "C C++ asm")

-x, --exclude=LANGS

exclude languages in LANGS -l, --lang-option=L:OPT pass OPT as a default for language L (see below)

-m, --lang-map=MAPFILE

use MAPFILE to map file names onto source language

-d, --default-lang=LANG

make LANG the default source language

-p, --prune=NAMES

exclude the named files and/or directories

--files0-from=F

tokenize only the files specified by NUL-terminated names in file F

--help

display this help and exit --version output version information and exit

The following arguments apply to the language-specific scanners:

C language:

-k,--keep=CHARS

Allow CHARS in single-token strings, keep the result

-i,--ignore=CHARS

Allow CHARS in single-token strings, toss the result

-u,--strip-underscore

Strip a leading underscore from single-token strings

C++ language:

-k,--keep=CHARS

Allow CHARS in single-token strings, keep the result

-i,--ignore=CHARS

Allow CHARS in single-token strings, toss the result

-u,--strip-underscore

Strip a leading underscore from single-token strings

Java language:

-k,--keep=CHARS

Allow CHARS in single-token strings, keep the result

-i,--ignore=CHARS

Allow CHARS in single-token strings, toss the result

-u,--strip-underscore

Strip a leading underscore from single-token strings

Assembly language:

-c,--comment=CHARS

Any of CHARS starts a comment until end-of-line

-k,--keep=CHARS

Allow CHARS in tokens, and keep the result

-i,--ignore=CHARS

Allow CHARS in tokens, and toss the result

-u,--strip-underscore

Strip a leading underscore from tokens

-n,--no-cpp

Don't handle C pre-processor directives

Text language:

-i,--include=CHAR-CLASS

Treat characters of CHAR-CLASS as token constituents

-x,--exclude=CHAR-CLASS

Treat characters of CHAR-CLASS as token delimiters

Perl language:

-i,--include=CHAR-CLASS

Treat characters of CHAR-CLASS as token constituents

-x,--exclude=CHAR-CLASS

Treat characters of CHAR-CLASS as token delimiters

-d,--dtags

Include documentation tags

Lisp language:

Report bugs to [email protected]

The full documentation for the id-utils package is maintained as a Texinfo manual. If the info and id-utils packages are properly installed, the command

• info 'ID database'

should give you access to the complete manual.