muchsync1

Usage

Overview

muchsync synchronizes the contents of maildirs and notmuch tags across machines. Any given execution runs pairwise between two replicas, but the system scales to an arbitrary number of replicas synchronizing in arbitrary pairs. For efficiency, version vectors and logical timestamps are used to limit synchronization to items a peer may not yet know about.

To use muchsync, both muchsync and notmuch should be installed someplace in your PATH on two machines, and you must be able to access the remote machine via ssh.

In its simplest usage, you have a single notmuch database on some server SERVER and wish to start replicating that database on a client, where the client currently does not have any mailboxes. You can initialize a new replica in $HOME/inbox by running the following command:

muchsync --init $HOME/inbox SERVER

This command may take some time, as it transfers the entire contents of your maildir from the server to the client and creates a new notmuch index on the client. Depending on your setup, you may be either bandwidth limited or CPU limited. (Sadly, the notmuch library on which muchsync is built is non-reentrant and forces all indexing to happen on a single core at a rate of about 10,000 messages per minute.)

From then on, to synchronize the client with the server, just run:

muchsync SERVER

Since muchsync replicates the tags in the notmuch database itself, you should consider disabling maildir flag synchronization by executing:

notmuch config set maildir.synchronize_flags=false

The reason is that the synchronize_flags feature only works on a small subset of pre-defined flags and so is not all that useful. Moreover, it marks flags by renaming files, which is not particularly efficient. muchsync was largely motivated by the need for better flag synchronization. If you are satisfied with the synchronize_flags feature, you might consider a tool such as offlineimap as an alternative to muchsync.

Options

-C file, --config file

Specify the path of the notmuch configuration file to use. If none is specified, the default is to use the contents of the environment variable $NOTMUCH_CONFIG, or if that variable is unset, the value $HOME/.notmuch-config. (These are the same defaults as the notmuch command itself.)

-F

Check for modified files. Without this option, muchsync assumes that files in a maildir are never edited. -F disables certain optimizations so as to make muchsync at least check the timestamp on every file, which will detect modified files at the cost of a longer startup time. If muchsync dies with the error “message received does not match hash,” you likely need to run it with the -F option.

Note that if your software regularly modifies the contents of mail files (e.g., because you are running offlineimap with “synclabels = yes”), then you will need to use -F each time you run muchsync. Specify it as a server option (after the server name) if the editing happens server-side.

-r /path/to/muchsync

Specifies the path to muchsync on the server. Ordinarily, muchsync should be in the default PATH on the server so this option is not required. However, this option is useful if you have to install muchsync in a non-standard place or wish to test development versions of the code.

-s ssh-cmd

Specifies a command line to pass to /bin/sh to execute a command on another machine. The default value is “ssh -CTaxq”. Note that because this string is passed to the shell, special characters including spaces may need to be escaped.

-v

The -v option increases verbosity. The more times it is specified, the more verbose muchsync will become.

--help

Print a brief summary of muchsync’s command-line options.

--init maildir

This option clones an existing mailbox on a remote server into maildir on the local machine. Neither maildir nor your notmuch configuration file (see --config above) should exist when you run this command, as both will be created. The configuration file is copied from the server (adjusted to reflect the local maildir), while maildir is created as a replica of the maildir you have on the server.

--nonew

Ordinarily, muchsync begins by running “notmuch new”. This option says not to run “notmuch new” before starting the muchsync operation. It can be passed as either a client or a server option. For example: The command “muchsync myserver --nonew” will run “notmuch new” locally but not on myserver.

--noup, --noupload

Transfer files from the server to the client, but not vice versa.

--upbg

Transfer files from the server to the client in the foreground. Then fork into the background to upload any new files from the client to the server. This option is useful when checking new mail, if you want to begin reading your mail as soon as it has been downloaded while the upload continues.

--self

Print the 64-bit replica ID of the local maildir replica and exit. Potentially useful in higher-level scripts, such as the emacs notmuch-poll-script variable for identifying on which replica one is running, particularly if network file systems allow a replica to be accessed from multiple machines.

--newid

Muchsync requires every replica to have a unique 64-bit identifier. If you ever copy a notmuch database to another machine, including the muchsync state, bad things will happen if both copies use muchsync, as they will both have the same identifier. Hence, after making such copy and before running muchsync to synchronize mail, run muchsync --newid to change the identifier of one of the copies.

--version

Report on the muchsync version number

Examples

To initialize a the muchsync database, you can run:

muchsync -vv

This first executes “notmuch new”, then builds the initial muchsync database from the contents of your maildir (the directory specified as database.path in your notmuch configuration file). This command may take several minutes the first time it is run, as it must compute a content hash of every message in the database. Note that you do not need to run this command, as muchsync will initialize the database the first time a client tries to synchronize anyway.

muchsync --init ~/maildir myserver

First run “notmuch new” on myserver, then create a directory ~/maildir containing a replica of your mailbox on myserver. Note that neither your configuration file (by default ~/.notmuch-config) nor ~/maildir should exist before running this command, as both will be created.

To create a notmuch-poll script that fetches mail from a remote server myserver, but on that server just runs notmuch new, do the following: First, run muchsync --self on the server to get the replica ID. Then take the ID returned (e.g., 1968464194667562615) and embed it in a shell script as follows:

#!/bin/sh
self=$($HOME/muchsync --self) || exit 1
if [ "$self" = 1968464194667562615 ]; then
    exec notmuch new
else
    exec $HOME/muchsync -r ./muchsync --upbg myserver
fi

The path of such a script is a good candidate for the emacs notmuch-poll-script variable.

Alternatively, to have the command notmuch new on a client automatically fetch new mail from server myserver, you can place the following in the file .notmuch/hooks/post-new under your mail directory:

#!/bin/sh
muchsync --nonew --upbg myserver

Files

The default notmuch configuration file is $HOME/.notmuch-config.

muchsync keeps all of its state in a subdirectory of your top maildir called .notmuch/muchsync.