mirror-master1

Usage

Overview

mm is the mirror master. It runs multiple calls to mirror in parallel and handles locking and minimal retry times.

As with mirror, mm is a package written in Perl.

mm presumes that all the package details are stored in a directory called packages. All the packages for a given site must be in a file whose name is the same as that of the site. So in packages/sunsite.org.uk will be all the package details for the host sunsite.org.uk.

Options

-t

Ignore the minimal retry timers.

-o site:package

Only mirror the given site:package.

-debug

Enable debugging. If this argument is given more than once the debugging level will increase. Currently the maximum useful level is four.

-s

Enable status debugging. This will print out the status of any subprocess as it is spawned or as it exits.

Configuration file

The configuration file is parsed as a series of statements. Blank lines and lines beginning with a hash are ignored. There are are two kinds of statements, controls and site:package details.

Control statements are of the form:

keyword=value

You can add whitespace around the equals.

Here is a list of the keywords and their values, any defaults are given inside square brackets:

home

Directory that mm chdirs to before doing any work. ['']

max

Maximum number of mirrors that can be running in parallel at any one time ['6']

mirror

This is the way to call mirror. Any occurence of $args, $package or $site is replace with the appropriate entry from the package. Any $pkg is replaced with a version of $package with all characters that cause problems in filenames converted to underscore. The default is: exec ./mirror \$args -p'\$package' packages/\$site > logs/\$site:\$pkg 2>&1 ['']

cmd

Run the given command. This can occur multiple times. ['']

cmdin

Run the given command and use its output as mm input. This can occur multiple times. ['']

skip

Skip the given site:package when it occurs. ['']

In addition to control statements there are also package details. Each package details statement is of the form:

site:package min-restart-last-ok min-restart-last-notok mirror-args

The site and package fields are matched against those in the packages. A mirror will only be run to this site:package if the last mirror run was more than min hours out. The min-restart-last-ok limit will be used if the last call to mirror was successful. If the mirror call failed then min-restart-last-notok will be used.

Examples

Here is a simple mm input file to manage mirroring two packages.

home=/public/mirror
ftp.cl.cam.ac.uk:JIPS-nrs 20 20
hcrl.open.ac.uk:hcrl-psion 20 20

More usually the auxilly program pkgs_to_mmin will be used to automatically convert the contents of the packages directory into the site:package statements.

home=/public/mirror
cmdin=./pkgs_to_mmin packages/*