expire8

Usage

Overview

expire scans the history(5)-format text file pathdb/history and uses the information recorded in it to purge itself of old news articles. Its behaviour depends on the setting of groupbaseexpiry in inn.conf.

When groupbaseexpiry is false, article expiration is primarily done by expire based on the expiration rules in expire.ctl that match the storage class of each article. The articles and the history entries are removed by expire, and then expireover does the additional cleanup of removing the overview database entries. History entries of expired articles are removed only if they are older than the number of days specified in the "/remember/" line in expire.ctl. Articles stored using a storage method that has self-expire functionality like CNFS are by default not affected by expire's primary behaviour (but see the -N flag to disable this).

When groupbaseexpiry is true, article expiration is primarily done by expireover based on the expiration rules in expire.ctl that match each newsgroup. Articles are removed from the news spool by expireover, and then expire does some additional cleanup to remove old history database entries.

For articles in self-expiring storage methods when groupbaseexpiry is set to false in inn.conf and the -N flag is not given, or for all articles when groupbaseexpiry is set to true, expire.ctl is ignored except the "/remember/" line; expire then only probes to see if the article still exists, and purges the relevant history entries if appropriate.

Regardless the setting of groupbaseexpiry, expireover should be run along with expire, usually via news.daily out of cron.

Note that expire never purges articles which do not match any entry in expire.ctl.

Also note that if groupbaseexpiry is true, the server needs an overview database in order to expire articles in storage backends that are not self-expiring. If you do not plan to have an overview database, it would then be better to only use self-expiring backends like CNFS, as the history entries will still get cleaned up by expire when it detects that an article no longer exists in that backend.

Options

-d dir

If the -d flag is used, then the new history file and database is created in the specified directory dir. This is useful when the filesystem does not have sufficient space to hold both the old and new history files. When this flag is used, expire leaves the server paused and creates a zero-length file named after the new history file, with an extension of ".done" to indicate that it has successfully completed the expiration. The calling script should install the new history file and unpause the server. The -r flag should be used with this flag.

-f file

To specify an alternate history file, use the -f flag. This flag is valid when used with the -d flag, and the output will be written to the specified file. The default without -f is "history".

-g file

If the -g flag is given, then a one-line summary equivalent to the output of -v 1, except preceded by the current time, will be appended to the specified file.

-h file

To specify an alternate input text history file, use the -h flag. expire uses the old dbz database to determine the size of the new one. (If the -d flag is not used, the output filename will be the same as the input filename with an extension of ".n".)

The default without the -h flag is pathdb/history.

-i

To ignore the old database, use the -i flag.

-N

The control file is normally ignored for articles in storage methods which have self-expire functionality. If the -N flag is used, expire still uses the control file for these articles.

This parameter is only useful when groupbaseexpiry is set to false in inn.conf.

-n

If innd is not running, use the -n flag and expire will not send the "pause" or "go" commands. (For more details on the commands, see ctlinnd(8)). Note that expire only needs exclusive access for a very short time -- long enough to see if any new articles arrived since it first hit the end of the file, and to rename the new files to the working files.

-p

expire makes its decisions on the time the article arrived, as found in the history file. This means articles are often kept a little longer than with other expiration programs that base their decisions on the article's posting date. To use the article's posting date, use the -p flag.

-r reason

expire normally sends a "pause" command to the local innd daemon when it needs exclusive access to the history file, using the string "Expiring" as the reason. To give a different reason, use the -r flag. The process ID will be appended to the reason. When expire is finished and the new history file is ready, it sends a "go" command. See also the -n flag.

-s size

expire determines the optimal size of the new history file from the size of the old one. In case you want to force a specific size, use this flag to optimize the new history database for approximately size key-value pairs (i.e. lines in history). Accurately specifying the size will create a more efficient database. (The size should be the estimated eventual number of articles, typically the size of the old history file, in lines.)

-t

If the -t flag is used, then expire will generate a list of the tokens that should be removed on its standard output, and the new history file will be left in history.n, history.n.dir, history.n.index and history.n.hash. This flag is useful for debugging when used with the -n flag. Note that if the -f flag is used, then the name specified with that flag will be used instead of history.

-v level

The -v flag is used to increase the verbosity of the program, generating messages to standard output. The level should be a number, where higher numbers result in more output. Level one will print totals of the various actions done (not valid if a new history file is not written), level two will print a report on each individual file, while level five results in multiple lines of output for every history line processed.

-w number

Use the -w flag to "warp" time so that expire thinks it is running at some time other than the current time. The value should be a signed floating point number indicating the number of days to use as the offset.

-x

If the -x flag is used, then expire will not create any new history files. This is most useful when combined with the -n and -t flags to see how different expiration policies would change the amount of disk space used.

-z file

If the -z flag is used, then articles are not removed, but their names are appended to the specified file. See the description of delayrm in news.daily(8). If a filename is specified, it is taken as the control file and parsed according to the rules in expire.ctl. A single dash ("-") may be used to read the file from standard input. If no file is specified, the file pathetc/expire.ctl is read.

Tombstone log

When expiretombstone and groupbaseexpiry are enabled in inn.conf, expire consumes the per-cycle deletion log produced by expireover, innd, and sm -r (expireover.tombstone and cancels.tombstone log files in pathdb) so it can drop history entries for those articles without doing a per-article "SMretrieve(RETR_STAT)" syscall. An empty tombstone is treated as "no cancels this cycle" and the slow scan is skipped entirely. See inn.conf(5) under expiretombstone for the file lifecycle, locking model, and recovery story.