couriermlm: The courier mailing list manager

DESCRIPTION

couriermlm is the Courier mail server's mailing list manager. This command sets up, maintains, and manages mailing lists. couriermlm automatically handles requests to subscribe and unsubscribe list members, detects undeliverable addresses and removes them from the subscription rolls. Mailing lists managed by couriermlm require zero human administrative oversight. couriermlm supports digests, write-only posting aliases, and moderated mailing lists.

CREATING A MAILING LIST

Anyone can use couriermlm, not just the system administrator. The Courier mail server mail server translates an address list-name@domain as a local address with a corresponding \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 file. Anyone that can install a \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 file, and can schedule cron(8) jobs, can run a couriermlm mailing list.

Note that the system administrator can optionally remove the \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 support from the the Courier mail server. couriermlm will not work in that case.

Setting up a mailing list consists of the following steps:

Run couriermlm create

Use this command to create a directory where couriermlm keeps all mailing list related files.

Configure the mailing list

The couriermlm create command initializes the mailing list subdirectory with some default template responses. It is necessary to customize them for your mailing list, and it may be necessary to issue some additional commands in order to configure appropriate mailing list options -- such as enabling unrestricted posting privileges, and enabling moderation.

Create \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 files

Set up to run couriermlm to distribute mailing list messages, and process requests.

Set up cron(8)

You need to set up cron(8) jobs to run the couriermlm hourly and couriermlm daily commands, which perform regular mailing list maintenance.

Back up subscription lists

As part of your daily job you should also run the export command, in order to back up the mailing list subscriber information. In the event that the mailing list database gets corrupted or lost, you can restore it from this backup file. See the export command for more information.

Note

Setting up a digest for the mailing list requires additional steps. See "\m[blue]Setting up a mailing list digest\m[]\s-2\u[2]\d\s+2" below for more information.

The first step is to run the following command:

couriermlm create directory ADDRESS=list@domain

directory is the mailing list directory that will be managed by couriermlm. This directory should not be created in advance, this command creates this directory, and initializes it.

list@domain is the mailing list's address, the address that sends messages to the mailing list.

An additional setting, URL may also be specified:

couriermlm create directory ADDRESS=list@domain URL=url

“url” would be the URL of the WebMLM web page for this mailing list. See \m[blue]webmlmd(1)\m[]\s-2\u[3]\d\s+2 for more information.

The directory created by couriermlm create is initialized to contain a number of text files that couriermlm sends back as replies to administrative commands. It is necessary to edit these template files and adjust the text in those files for this mailing list. All template filenames end with .tmpl, and their contents are self explanatory. Some important template files are:

help.tmpl

This text is returned in response to the help command. This text must be modified depending upon whether this mailing list is a moderated mailing list, has a digest version, or if any other non-default configuration options are set for the mailing list.

sub.tmpl

This is the reply that's sent back in response to a subscription request. Less important is unsub.tmpl, which is the response to a request to unsubscribe.

sub2.tmpl

This is the successful subscription confirmation. A brief overview of the mailing list might be appropriate here.

couriermlm has rudimentary support for non-English templates. The couriermlm create command has a single, optional parameter, --lang=suffix The option must be specified after the mailing list directory's name:

couriermlm create directory --lang=es ADDRESS=list@domain URL=url

The templates for the stock text files come from /usr/lib/courier/couriermlm. This directory contains all the *.tmpl and *.html template files that couriermlm create installs in the new mailing list directory.

Translated versions of template files are installed in the same directory, /usr/lib/courier/couriermlm. The translated version of file must be installed as file.suffix, for example: help.tmpl.es is the translated version of help.tmpl, with the “es” suffix.

The --lang=suffix option installs the the .suffix version of each text template file, if available. If not, the stock English template file gets copied, as usual.

Note

Do not remove the English template files, even if you never use them. They must be left in place, in /usr/lib/courier/couriermlm, for couriermlm create to work correctly.

Example:

couriermlm create /home/lists/users-list --lang=es ...

This example installs the *.es versions of mailing list template files. Only the stock, English template files come with the Courier mail server. This mechanism only provides an easy integration ability with template files from other sources.

There are some additional configuration files that can be modified to suit your taste:

headeradd

This file can be initialized to contain any mail headers that will be automatically added to every mailing list message. The contents of this file are simply prepended to every message that goes out. Blank lines are not allowed.

headerdel

This file lists any headers that will be automatically removed from every mailing list message before it's sent. List each header one per line, including the : character. For example, to remove all Received: and Date: headers from every message, initialize this file to contain the following two lines:

Received:
Date:

Both the headeradd and headerdel files can be used to implement a popular feature of setting the replies to every message to go to the mailing list. Having "Reply-To:" in headerdel, removes any existing Reply-To: header, and then having "Reply-To: list@domain" in headeradd appends a fixed Reply-To: header to every message.

The create command also creates the following subdirectories in the mailing list directory:

sublist

This subdirectory has the database files that contain the mailing list's subscription list.

unsublist

This subdirectory stores files that contain information about addresses that have been unsubscribed from the mailing list. This information might be of some use when tracking down an old subscription. The contents of this directory are not automatically purged, you must set up your own purging mechanism for this directory.

commands, commands.dat

These directories store temporary files that contain pending (unconfirmed) commands for the mailing list manager. The couriermlm commands hourly and daily must be executed regularly in order to periodically purge stale entries.

modqueue

Messages awaiting moderator approval (for moderated lists).

MANUAL COMMANDS

couriermlm may also be run manually from the command line as follows:

/usr/bin/couriermlm command directory [ options... ]

command is a command from the following list. directory is the mailing list directory. The commands are:

create

Create a mailing list.

update

Update/restore mailing list templates. The original, default, mailing list message template files (*.tmpl and *.tmpl.html) are reinstalled into the mailing list directory. This command must be processed for every mailing list directory after upgrading to the Courier server version 0.55, or later, from earlier versions (but see below). couriermlm in version 0.55 of the Courier mail server uses updated templates files, which must be installed in every mailing list directory. Although the names of many template files have not changed, the embedded markup codes in the template files work differently.

Begin by making a backup copy of all *.tmpl files in the existing couriermlm mailing list directory.

Run the “update” command on the list directory.

Review the backed up template files, identify your customized changes, then retype them into new template files installed by the “update” command.

Proceed to the next mailing list directory.

Yes, this is going to be a pain. This is the first overhaul of couriermlm's infrastructure in many years. Once a decade, some elbow grease must be sacrificed in the name of progress. It's not the end of the world.

In an extreme emergency, preserve the couriermlm from the previous version of the Courier mail server. Mass-update all existing lists' .courier-list and .courier-list-default files to run the old couriermlm. Then, migrate each mailing list on a predetermined schedule. After migrating each list, put the default path back into the list's .courier files.

Keep in mind the following issues, while migrating the lists:

The help.tmpl template file is usually the one that gets customized the most. In most cases, large bits and pieces of this file, that document certain list options that do not apply to this list, must be removed.

There are several new template files, including a number of *.html files that refer to new HTML-formatted responses from couriermlm, and the WebMLM interface.

Be aware of updated mail headers declared in several template files. The MIME character set is now given as utf-8. Some templates no longer contain the Mime-Version and Subject headers. The Subject headers are moved to separate template files. Many existing template files now contain both a plain text and an HTML-formatted version of the couriermlm response. Always scroll to the end of each template file, to reveal any appended HTML portion of template's text.

set

Set mailing list options.

sub

Manually subscribe an address to the mailing list.

unsub

Manually unsubscribe an address from the mailing list.

lsub

List all the subscribers to this mailing list.

laliases

List write-only aliases for this mailing list.

export

Export mailing list subscriber information.

import

Import mailing list subscriber information.

ctlmsg

Receive and interpret a control message.

info

Display a subscription record.

msg

Post a message to the mailing list.

hourly

Perform hourly maintenance. It is necessary to set up a cron(8) job to execute the hourly command once an hour.

daily

Perform daily maintenance. It is necessary to set up a cron(8) job to execute the daily command once a day.

digest

Create a digest. See "\m[blue]Setting up a mailing list digest\m[]\s-2\u[2]\d\s+2" below for more information.

MANUAL SUBSCRIPTION MANAGEMENT

The sub, unsub, lsub, laliases, export, and import commands allow manual subscription list management. Normally, subscription-related commands are done by sending an appropriate mailing list command, see "\m[blue]Mailing List Commands\m[]\s-2\u[4]\d\s+2", below.

couriermlm sub directory user@domain

This command adds the address <user@domain> to the subscription list. couriermlm will now read a free-form comment or a note from standard input, terminated by an end-of-file (usually CTRL-D). The free-form comment is stored in the subscription database, together with the address, and is shown by the "info" command.

couriermlm unsub directory user@domain

This command remove the address <user@domain> from the subscription rolls. couriermlm will also read a free-form comment, which is added to the subscription record. After removing this address from the subscription rolls, its subscription record is archived in the directory/unsublist directory.

couriermlm lsub directory

This command lists all the addresses subscribed to the list, on standard output, one per line.

couriermlm laliases directory

This command lists all write-only aliases that have been subscribed to the list, together with the subscriber address that added each alias. See "\m[blue]Write-Only Aliases\m[]\s-2\u[5]\d\s+2" for more information.

couriermlm export directory

The export command lists the contents of the subscription database on standard output. The export command produces the following output format:

address
subscription information
 ...
address
subscription information
 ...

"address", is an address subscribed to the mailing list. This is followed by its corresponding subscription information, usually a copy of the subscription request that was used to add the address to the mailing list. The subscription information is terminated by a line containing a single period. Any lines in the subscription information that begin with a period have an extra period prepended to them.

couriermlm import directory

The import command reads on standard input a previously exported mailing list subscription database, and adds those addresses to the indicated mailing list.

It is highly recommended to make a regular backup of subscriber information using the export command, in the event that the subscription database gets corrupted. In which case the import command can be used to rebuild the subscription database, in absence of any direct backups of the database files.

SETTING MAILING LIST OPTIONS

The set command sets various list options:

couriermlm set directory option=value option=value...

Note

Setting the ADDRESS or the URL options, described below, automatically updates the contents of the headeradd configuration file. Its existing “List-” headers are removed and replaced by updated “List-” headers that reflect the revised list address or URL.

One or more options can be set with the same command. The available options are:

ADDRESS=address

The base E-mail address for this mailing list.

URL=address

The URL of WebMLM web page for this mailing list.

Note

WebMLM determines its own URL automatically, by reading its HTTP headers. This setting is used by couriermlm, which has no knowledge of the web server's configuration, and needs to know the correct URL to insert into generated messages.

CASESENSITIVE=flag

If flag is "1", the userid portion of E-mail addresses are case-sensitive. The domain address portion is always case-insensitive. The default setting is "0" making both userid and domain address portions of E-mail addresses case-insensitive.

Note

Be careful when changing this setting on an active list. Changing this option to “CASESENSITIVE=1”, forces all existing subscribers to make sure their addresses are lowercase-only, otherwise they will no longer be recognized as subscribers. Changing this option to “CASESENSITIVE=0” blocks all subscribed addresses that have uppercase characters in the userid portion of their E-mail address. They will continue to receive mailing list traffic, but unable to post messages to the list, or unsubscribe from it. It will not be possible to unsubscribe those addresses even by running the couriermlm command manually.

DIGEST=directory

Enable digests. directory is the pathname to the previously-createddigest list directory. See "\m[blue]Setting up a mailing list digest\m[]\s-2\u[2]\d\s+2" below for more information.

KEYWORD=keyword

Set the subject line keyword for mailing list messages. If set, couriermlm inserts "[keyword]" into the subject of every mailing list message, to aid sorting by the recipients.

MAXBOUNCES=n

Maximum number of bounce notifications sent by the hourly command, in order to prevent the mail system from being overloaded. The default is 20 bounce notifications. Any unsent notifications will be carried over to the next hourly job.

MAXMODNOTICES=n

Maximum number of moderation reminders sent by the hourly command, in order to prevent the mail system from being overloaded. The default is 20 moderation reminders. Any unsent reminders will be carried over to the next hourly job.

MAXFETCHSIZE=K

Maximum size, in kilobytes, of a response to the fetch command. The default is 100Kb. This option is used to minimize the impact of abusive requests for the entire archive, with a forged return address.

NAME=name

The name that's listed on the return address of administrative messages. Note that if name contains spaces, you should quote this argument in the shell. The default value is "Courier Mailing List Manager".

NOBOZOS=flag

If flag is "0" couriermlm will not attempt to block misdirected subscribes and unsubscribes that are sent to the mailing list's posting address. If flag is "1" (the default), those kinds of messages will be bounced appropriately.

NODSN=flag

If flag is "1" couriermlm will use a Delivery Status Notification setting of "never" when it sends confirmation requests and help messages: this should reduce the amount of useless failure notifications generated when couriermlm dutifully replies to spam received by the mailing list administrative addresses, i.e. -(un)subscribe and -help. If flag is "0" (the default), a DSN setting of "fail" will be used. Please see sendmail(1)'s -n option for more details on the DSN setting.

POST=option

Set posting options. option is one of three values: "subscribers" - only subscribers may post messages to this mailing list (this is the default); "all" - anyone can post messages to this mailing list; "mod" - only subscribers may post, and messages are sent to the list owner for approval (moderation).

POSTARCHIVE=option

Set access to archived messages. option is either: "all" - Anyone can access the mailing list archive; or "subscribers" - only subscribers can access the archive. The default is "all".

PURGEARCHIVE=d

Purge archived mailing list messages after d days. The default is 0 days - messages are never removed from the archive subdirectory.

PURGEBOUNCE=d

Wait d days for the probe message, that automatically unsubscribes undeliverable addresses, to bounce (default: 14 days). Probe messages are sent three days (default) after the first message to an address bounces.

PURGECMD=h

Purge unconfirmed subscribe/unsubscribe requests after h hours (default: 48 hours).

REMODERATE=h

Resend a moderation reminder after h hours (default: 12 hours).

REPORTADDR=address

Mail daily reports of new and removed subscribers to this address. Must be set in order to receive reports. Provide an empty address to stop reporting.

SIMPLECONFIRM=n

If n is 1, confirmation requests may be acknowledged without adding “yes” to the subject line.

Note

The text in sub.tmpl, unsub.tmpl, and help.tmpl may need adjusting.

SUBSCRIBE=option

Set subscription options. option is either "all", meaning that anyone can subscribe, or "mod", meaning moderated subscription requests, where all subscription requests are sent to the mailing list owner for approval. The default is "all".

STARTPROBE=n

Send a probe to a bouncing address n days after receiving the first bounce. Basically this means that an address must bounce for at least n days before it gets a probe message. The default is 3 days.

Option names and settings are case sensitive.

Note

If you set up a digest list, you MUST set identical POSTARCHIVE option for both the main list and the digest list.

DISPLAYING A SUBSCRIPTION RECORD

The info command displays the subscription record for the requested address:

couriermlm info directory user@domain

This displays the subscription record for "user@domain", which typically consists of a copy of the initial subscription request, and confirmation.

SENDING MESSAGES TO THE MAILING LIST

The msg commands reads an E-mail message on standard input, and mails the contents of the message to the mailing list's subscribers.

If the POST option is set to "subscribers", the message is rejected unless the address in its From: header is a subscriber to this mailing list.

Control files headeradd and headerdel are read, and are applied to the message, as described previously.

MAILING LIST COMMANDS

Mailing list commands can be sent via E-mail to couriermlm by sending a message to <list-command@domain>. The "default" \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 file runs couriermlm to receive mail for all addresses of this form.

couriermlm reads the DEFAULT environment variable, which is set by the Courier mail server, that indicates the specific command. The available commands are:

help

A simple autoresponder. couriermlm mails the sender the contents of the help.tmpl file.

A request to subscribe to this mailing list. couriermlm reads the sender's address in order to determine what address to subscribe.

subscribe-name=domain

Explicitly specify the address to subscribe to the mailing list, instead of using a return address. In the previous example, sending a message addressed to <[email protected]> would result in a subscription request for <john=domain.com>. Any unusual punctuation characters in the address must be replaced by a plus sign, followed by two hexadecimal digits that represent the punctuation character's ASCII code.

unsubscribe

A request to unsubscribe to this mailing list.

unsubscribe-name=domain

Explicitly specify the address to unsubscribe from the mailing list.

alias-subscribe

Set up a write-only alias (see below).

alias-subscribe-name=domain

Explicitly specify the subscriber address for which a write-only alias needs to be set up.

There are other commands that are used internally for maintaining the mailing list.

WRITE-ONLY ALIASES

Write-only E-mail aliases can send messages to the mailing list, but they do not receive any mailing list messages themselves. A write-only alias can be set up by any subscriber. Only one write-only alias is allowed per subscribed E-mail address. Write-only aliases are not needed for mailing list that has the POST=all option set.

To set up a write-only alias, the subscriber sends a couriermlmalias-subscribe command. The subscriber's E-mail address can be explicitly specified in a similar manner as the subscribe command.

The subject line of the E-mail message must contain the E-mail write-only alias to be set up, and nothing else. couriermlm responds with a confirmation request, just like when subscribing to the list. This request must be acknowledged in the same way.

A subscriber's write-only alias can be changed at any time by repeating this procedure. The new alias replaces the previous one. To prevent abuse, there's a limit of at most one alias-subscribe command every 30 minutes.

Leave the subject of the E-mail message blank in order to remove an existing write-only alias,

SETTING UP A MAILING LIST DIGEST

couriermlm supports mailing list digests. Mailing list digests are created as a second, separate, mailing list. The create command initializes a second mailing list directory, and then additional configuration takes place which links the main mailing list to the digest list.

If the mailing list address is [email protected], the address of the digest version of the mailing list is usually [email protected], but it doesn't have to be this address. The only requirement is that the directory for the digest version of the mailing list must reside on the same file system as the directory for the mailing list itself, and both must be owned by the same userid.

To set up a mailing list digest, first proceed with the steps to create the mailing list itself. After the mailing list is created and configured, proceed as follows:

Create the digest list directory

Execute the create command to create the digest version of the list:

/usr/bin/couriermlm create \
   /path/to/digest/list/directory \
   ADDRESS=[email protected]

Use the full pathname to the mailing list directory, and the address of the digest version of the mailing list.

Configure the digest list

Execute the set command to set any appropriate options for the digest list. There one important differences to note: messages are not posted to the digest list directly, so there is no moderation option, however the digest version of the list can have moderated subscription requests.

Link the two lists

Set the DIGEST option for the main mailing list, specifying the directory of the digest list. This keyword lets couriermlm know that a digest version is available.

Note

You MUST set identical POSTARCHIVE option for both the main list, and the digest list.

Create \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 files

It is necessary to create \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 files for the digest list just like the main list, except for some important differences, which are noted below.

Create cron(8) jobs

It is also necessary to create cron jobs for the digest list exactly like the main list, to run the hourly and daily cleanup. It's possible to set up one set of cron jobs to run hourly and daily cleanups consecutively for both lists.

Create a digest cron(8) job

The digest creates and distributes the digest version of the list. It can be executed by a cron(8) job, or the command can be executed manually.

The main mailing list is supported by three \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 files, as previously described: the posting address, the owner forwarding address, and the default address that handles administrative control messages. In the following example, the names $HOME/.courier-list, $HOME/.courier-list-owner, and $HOME/.courier-list-default are used to represent each one of these files, and the following names are used to represent the \m[blue]dot-courier(5)\m[]\s-2\u[1]\d\s+2 files that correspond to the digest version of the mailing list: $HOME/.courier-list-digest, $HOME/.courier-list-digest-owner, and $HOME/.courier-list-digest-default. Note, however, that the digest version of the mailing list can have any name, not necessary the name of the list, followed by "digest".

The contents of both $HOME/.courier-list$HOME/.courier-list-digest must be the same. Sending a message to the digest address should really end up sending a message to the main mailing list. Do not put the address of the digest mailing list directory in $HOME/.courier-list-digest, instead specify the address of the main mailing list directory. Just copy $HOME/.courier-list to $HOME/.courier-list-digest.

However, the contents of $HOME/.courier-list-digest-default must specify the directory of the digest version of the mailing list. The digest list is managed separately from the main list, it has its own subscriber list that is separate from the list of subscribers to the main list. $HOME/.courier-list-default can simply be copied to $HOME/.courier-list-digest-default, then the directory can be changed in the latter.

$HOME/.courier-list-owner may use the same mailing list owner address as $HOME/.courier-list-digest-owner, or it can specify a different address. The both the digest and the main mailing list can have the same mailing list owner/moderator, or have a different owner/moderator.

The following command must be executed in order to link the two lists together:

/usr/bin/couriermlm set \
    /path/to/main/list/directory \
    DIGEST=/path/to/digest/list/directory

Setting the DIGEST option on the main list lets couriermlm know that a digest version is available. The DIGEST option must either use an absolute pathname, or a pathname that's relative to the main list directory (NOT the current directory).

When the DIGEST option is set, messages are simultaneously distributed to the mailing list's subscribers, saved in the archive subdirectory of the main list, then placed in the modqueue subdirectory of the digest list. Digest list do not employ moderation -- any moderation must take place on the main list -- so the modqueue subdirectory is recycled to compile individual messages for the digest.

Finally, something needs to be done in order to actually distribute the digest to the digest list's subscribers. This is done by running the following command:

/usr/bin/couriermlm digest /path/to/digest/directory N H

This command creates a digest, and sends it out. The N and H arguments are optional. The digest is created only if there's at least N messages that are waiting to be sent in the digest list, or if the oldest message is at least H hours old. Both options default to 0, so the default behavior is to send a digest with all unsent messages.

Note that when the digest is created, ALL unsent messages are packaged into the digest, even if some messages are more recent than the time interval specified by the H option. A cron(8) job can be set up to run the digest command, or run it manually.

couriermlm automatically provides the From:, To: headers on a message digest. Additional headers may be specified by the headeradd file in the digest list directory. The headerdel file has no effect. Note that the individual messages in the digest are copies of the messages from the main mailing list, and thus have the headeradd and headerdel headers processed from the main mailing list directory.

couriermlm (1)

SYNOPSIS

DESCRIPTION

CREATING A MAILING LIST

MANUAL COMMANDS

MANUAL SUBSCRIPTION MANAGEMENT

SETTING MAILING LIST OPTIONS

DISPLAYING A SUBSCRIPTION RECORD

SENDING MESSAGES TO THE MAILING LIST

MAILING LIST COMMANDS

WRITE-ONLY ALIASES

SETTING UP A MAILING LIST DIGEST

BUGS

RELATED TO couriermlm…

AUTHOR

NOTES