Rsync friendly encryption
rsyncrypto [options] srcfile dstfile keyfile key
rsyncrypto [options] -r srcdir dstdir keysdir key
This manual page documents the rsyncrypto command.
rsyncrypto is a utility that encrypts a file (or a directory structure) in a way that ensures that local changes to the plain text file will result in local changes to the cipher text file. This, in turn, ensures that doing rsync to synchronize the encrypted files to another machine will have only a small impact on rsync's efficiency.
rsyncrypto will encrypt files using a symmetric block cipher (AES). Each file is encrypted using a unique key. The file key is stored in two locations. One is the "key" file, and the second is inside the encrypted file itself. The second copy is encrypted using a RSA public key, which can be shared for all encrypted files.
Decrypting the file can take place in one of two forms:
Warm Decryption
keyfile is used to decrypt srcfile into dstfile.
Cold Decryption
If keyfile is lost and key is the private key for the RSA encryption, extract and create keyfile from the encrypted srcfile, and create both keyfile and dstfile.
If both keyfile and the private key are not available, decryption cannot take place. Typically, this means that you would either want to transfer the encrypted files in whatever way (secure or insecure, but probably rsync :), and keep the plain text file, as well as the key file, secure. As long as you saved the RSA private key, you can reconstruct them both.
The parameters meaning depend on the existence of the -r and -d flags.
srcfile or srcdir
This is the source for the operation. This will be the plain text file for encryption, and the cipher text file for decryption. If the -r option is given, srcdir is the directory from which files are to be encrypted/decrypted. Otherwise, srcfile is the file to be encrypted/decrypted. In any case, src must exist prior to running rsyncrypto.
dstfile or dstdir
Similar to src, this is the destination of the operation. It will be the plain text to be created if -d was given, or the encrypted file otherwise.
keyfile or keydir
The file holding the symmetric key for the encryption, or a directory in which symmetric key files reside. If the later, the key files will have the same name as the plain text files.
key
This file can either be a X509 certificate holding a public key, or a PEM format private key. The private key is never needed unless a cold decryption is required. See the req(1) or x509(1) manual pages for more info.
If dst does not exist, it will be created. The directory in which it is to be created must exist prior to running rsyncrypto.
If encrypting, and a symmetric key for the file is found, the same parameters will be used for the encryption unless the --fr or --fk options are used. If that's the case, the key file is going to be replaced with the correct parameters.
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-').
-h, --help
Show summary of options.
-V, --version
Print out the current version number.
-d, --decrypt
Decrypt src into dst. The default behavior is to encrypt src into dst, and create keyfile if it does not exist.
-r, --recurse
Recursively encrypt/decrypt an entire directory structure. All files under srcdir will be encrypted to files of the same name and path (almost, see --trim) under dstdir. If not already there, a key file will be created for each file under keydir.
--filelist
The first argument is a pointer to a file (or "-" for stdin) that has a list of files to process. Any name ending with a "/" is taken to be a directory in recursive mode, and any other name is a file name. The second argument is a directory where the files are encrypted to. --filelist and -r are mutually exclusive.
-c, --changed
Check the src and destination modified time (see --modify-window) prior to performing the operation. If the modified time is the same, assume that the files are also the same, and skip the operation. Notice that unlike rsync, file sizes are not compared. This is because the file size is expected to change during the encryption (and thus decryption) operation.
--risky-writes
Usually, when rsyncrypto writes a file, it uses a temporary name and moves the file to its final name when successfully done. This serves two purposes. During encryption, this prevents the false sense of security that may arise in case of an rsyncrypto interruption by preventing a partial file from being present (and in case of an update, by keeping the old backup). This also ensures that restarting rsyncrypto will enable it to correctly detect whether the operation needs to be repeated. During decryption, this prevents overwriting a file until we are certain we can decrypt its source.
The main problem with this approach is that, if the file being written is large, the file system needs to have enough space for two copies of it for the duration of the operation. Also, if hard links are maintained, the default rsyncrypto behavior will break the link.
Specify --risky-writes to make rsyncrypto directly overwrite the existing file using the final name.
--modify-window=num
Only applicable when -c is in effect. When comparing file modification times, treat as equal modification times that are num seconds apart. Default value is zero.
This option is most useful when the encrypted file is stored on a FAT file system. FAT can only store modification time with a 2 seconds accuracy. This may result in the modification time changing between original and encrypted file. In such a case, --modify-window=1 should be used.
--export-changes=logfile
Whenever encrypting or deleting a file, write the file's name to logfile. This allows passing logfile to rsync(1) using the --include-from rsync option.
This option is mostly useful if rsyncrypto is used to repeatedly encrypt a huge directory tree, and then using rsync to synchronize it remotely. The use of this option allows focusing rsync on only those files that are known to have changed, rather than have it send the entire file list to the remote machine, making the rsync operation faster. It should be noted that rsync version 3 starts the actual file transfer while it is still transferring the file list, making the performance gain from this option less noticeable.
When using this option in conjunction with rsync, keep in mind that logfile is zeroed out before being written into. This means that unless there is at least one completely successful rsync operation for each rsyncrypto invocation, it is possible to have files that have been changed on disk by a previous invocation of rsyncrypto, but were over written by a later invocation and do not show up in the latest version of logfile. In other words, if rsyncrypto is run more than once between two completely successful invocations of rsync, it is possible for logfile not to be complete. Use this option at your own risk!
It is highly recommended that you upgrade to rsync version 3 or higher rather than rely on this option.
-n, --name-encrypt=translation_file
Encrypt the file name. If this option is specified, this means that file names are encrypted as well. This is done by changing the name of the encrypted file to a random string of characters, and storing the translation in the file given as parameter. If working in recursive mode, this file is automatically also included in the encryption, and is given the fixed name "filemap". This is done so that decryption of the file names is possible when the symmetric keys are lost.
--ne-nesting=num
When the --name-encrypt option is used, by default all mangled file names are stored in a single directory. When encrypting large directory trees, this can result in rather big directories. Using "--ne-nesting" will split the files to sub directories, according to random bits that are part of the encrypted file name. "n" is the number of directories to nest.
--trim=num
Determine how many directory levels to trim from the beginning of the srcdir path when creating directories under dstdir. The default value is 1. See THE TRIM OPTION for more details.
--delete
When in recursive mode, delete files and directories under dstdir that do not exist under srcdir.
--delete-keys
When deleting files, also delete their symmetric key and file name mapping (if applicable). When only --delete is used, the symmetric key and file mapping are kept, so that if the file is recreated, it will be rsyncable with the original file. This option implies --delete.
-bnum --keybits=num
Specify the block cipher's key size. For AES, these can be 128, 192 or 256. Note that unless the --fr or --fk\P options are also specified, this will only affect files for which no keyfile exists.
--noatime=num
(Only on some platforms, most notably Linux). The platform usually keeps track of when files were last being accessed. Since rsyncrypto is typically used for backup, it opens the files it encrypts with the O_NOATIME option, which is a Linux specific extension that tells the system not to update the file's access time. When opening a file where the opening process is not owned by the file owner, an attempt to open the file with O_NOATIME will cause the open to fail with "permission denied".
num can be one of these options: 0 means do not use O_NOATIME at all. 1 (the default) means attempt to only use O_NOATIME if rsyncrypto thinks it will not cause a permission denied error. 2 means to use O_NOATIME always.
--no-archive-mode
Do not try to preserve timestamps. This option is incompatible with -c.
--gzip=path
By default, rsyncrypto will compress the plain text prior to encrypting it. In order not to become rsync unfriendly due to the compression, rsyncrypto will search the path for an instance of gzip(1) that supports the --rsyncable option. If the system's default gzip does not support this option (all Debian and derivatives, as well as some others, support it), use this option to tell rsyncrypto to use a different instance of gzip. The tests directory of rsyncrypto's source has a file called "gzip", that does NULL compression by redirecting the input and output to cat(1).
--roll-win=num, --roll-min=num, --roll-sensitivity=num
Affects the rolling block algorithm. See NOTES.
--fk, --fr
If command line, or a version with different defaults, dictate different values for the --roll-* options or the -b option, these will only affect files for which keyfile does not yet exist. specifying the --fk or --fr will recreate keyfile if it has values different than those in the previous key file.
-v, --verbose
Increase verbosity level. Specify several times to be more verbose.
encrypting stdin
If srcfile is given as `-', the plain text data will be read fro stdin. This does not yet work for decompression (see BUGS).
--roll-win=num, --roll-min=num, --roll-sensitivity=num
These values affect some of the inner workings of rsyncrypto's decision making. In particular, these affect how rsyncrypto decides when to revert to the IV.
These options should not be played with unless you understand what you are doing. They are not dangerous, in that they will never cause data loss. Any value should still allow decryption of the file. They do have an affect on rsync's efficiency.
These options are only there to allow quick response in case a security problem pops up with their default values. If that happens, an advisory may come out giving new values to fill in as a workaround. Until that happens, most users are advised to leave these options alone.
When running rsyncrypto in recursive mode, the directory structure under srcdir is re-created under dstdir, with one directory stripped from the path. In other words, if we have a directory structure which has:
a/b/c/file
running rsyncrypto with srcdir of "a/b", and dstdir of "f" will create "f/b/c/file".
The --trim options lets the user say how many parts to trim from srcdir when creating directories under dstdir and keydir. If, in the above example, we said --trim=0 then "f/a/b/c/file" would have been created. Likewise, if we said --trim=2 then "f/c/file" would have been created.
It is an error to give a trim value which is higher than the number of directory parts actually in srcdir. In the above example, --trim=3 would result in an error.
The roll values affect, to a not totally known degree, how much information is "leaked" between plain text and cipher text. The default values are supposed to leak less than 20 bits of aggregated information about the plain text file for every 8KB, in the worst case. These values, we hope, will be considered secure enough for most usages. However, if your country's fate is at stake, use a non-rsync optimized encryption, such as the one provided by openssl(1).
Before starting to use rsyncrypto you will need a public key in X509 certificate format. The quickest way to generate one of these is using the openssl(1) package. To generate a new 1536 bit RSA key as "backup.key", and store the corresponding unencrypted public key in "backup.crt", use the following command:
openssl req -nodes -newkey rsa:1536 -x509 -keyout backup.key -out backup.crt
It does not matter what you answer for the certificate details, or when you set it to expire. Rsyncrypto ignores everything in the certificate but the actual key.
To encrypt file "data" into "data.enc", storing the file's specific key at "data.key", run:
rsyncrypto data data.enc data.key backup.crt
Anywhere in these examples that "backup.crt" is used, "backup.key" can be used as well. The above could also have "backup.key" as the last argument. If these examples list "backup.key" specifically, then it means that "backup.crt" cannot be used instead.
"data.enc" will be created. If "data.key" exists from a previous run, then the newly created "data.enc" will have the same symmetric key and other attributes, thus maintaining the rsync friendliness of the encryption. It is therefor important to provide any previous key file for the data if one is available.
The command line to decrypt the file depends on whether you have the key file used when the file was encrypted. If you do, then decryption can take place using only the public key, with the following command:
rsyncrypto -d data.enc data.dec data.key backup.crt
If the key file was lost, the private key must be used to recover it from the encrypted file using the following command:
rsyncrypto -d data.enc data.dec data.key backup.key
"data.key" will be created by the above command, as well as "data.dec".
Use of rsyncrypto to encrypt whole directories is similar, with directory names replacing file names in the above example. To encrypt the content of a directory called "data" to a directory called "enc", storing the symmetric keys in a directory called "keys", run the following command:
rsyncrypto -r data enc keys backup.crt
As before the directories "enc" and "keys" will be created if they do not already exist. The directory structure under "plain" will be duplicated under "enc" and under "keys". That is, if there is a file called "foo" under "data", an encrypted files called "foo" will be created under "enc", and a key file named "foo" will be created under "keys". In other words, the first part of the path to "data/foo" (the part that says "data") is eliminated from the path, and this determines the name of the files under "enc" and "keys".
If a different trimming is required, the --trim option can be used. To have rsyncrypto create "enc/data/foo" instead of the above, use the following command:
rsyncrypto -r --trim=0 data enc keys backup.crt
Instead of taking all files from a single directory, it is possible to supply rsyncrypto with a list of files and directories which to encrypt. This is done using the --filelist option, like so:
rsyncrypto --filelist --trim=0 list enc keys backup.crt
"enc" and "keys" retain the same meaning as before. "list" is a text file with a list, one per line, of files to encrypt. The --trim=0 option says that if list contains a line saying "/home/foo/bar" which states the name of a file, then an encrypted file called "enc/home/foo/bar" will be created. Without it, the file will be called "enc/foo/bar". Likewise, with --trim=2, the file would be created as "enc/bar".
It is often a problem that the file names are not encrypted. If that is the case, we can ask rsyncrypto to encrypt the file names, as well as the actual files. Rsyncrypto does not actually encrypt the file names. Instead, it replaces the actual file name with a random series of characters. The mapping between the actual file name and the garbled name is stored in a mapping file.
rsyncrypto --name-encrypt=map --trim=0 -r plain enc keys backup.crt
This will encrypt the "plain" directory structure into enc, exactly as before. Unlike before, all files will be directly under the "enc" directory, and will be named as a meaningless garble. A file called "map" will be created under the current directory that contains a map between the gabled and the real name of the file. In addition, two files, called "enc/filemap" and "keys/filemap", will also be created. These are the encrypted and key file corresponding to the "map" file mentioned above. During decryption, if "map" does not exist, it will be created by decrypting "enc/filemap".
Important note: When performing file name encryption, it is vitally important to give the correct --trim values, both during encryption and during decryption. Failing to do so will cause rsyncrypto to search for non-existing file names, and to fail to locate the files to decrypt.
It is not yet possible to decrypt using password protected private key.
Decrypting from stdin is not yet supported.
rsyncrypto was written by Shachar Shemesh for Lingnu Open Source Consulting ltd. http://www.lingnu.com
This manual page was written by Shachar Shemesh <[email protected]>
http://rsyncrypto.lingnu.com