Encrypted Backups using rsync and duplicity with GPG and SSH on Linux/BSD

To prevent data-loss, it is crucial to use a backup system. Backups ensure that after a loss of data, this data can be recovered. Depending on the method, data can be recovered to either a late version or a predefined time.

In Linux/BSD operating systems, backups can be implemented by using several methods and programs. The combination of rsync and duplicity with GPG and SSH tools is a good choice because it fulfills the following needs:

Encryption and signing
Full and incremental backup
Network-capable
Compression

To set up the backup system a client-server system is required.

Server

We used a Raspberry Pi B+ with FreeBSD 10.1 as synchronization server system. But you are free to use any other Linux/BSD based system which is network-capable and where rsync can be installed.

~ uname -a
FreeBSD raspberry-pi 10.1-STABLE FreeBSD 10.1-STABLE #0 r282689: Sun May 10 07:19:57 UTC 2015
root@releng1.nyi.freebsd.org:/usr/obj/arm.armv6/usr/src/sys/RPI-B  arm

On the server-side system (in our case the Raspberry Pi) the fast and versatile file-copying tool rsync needs to be installed. rsync is typically used to remotely synchronize a file or files between two systems. It can also operate in daemon mode, serving files in the rsync protocol. On FreeBSD systems rsync can be installed by using the Ports-Collection.

~ cd /usr/ports/net/rsync
~ make install

After that, open the rsync-daemon runtime configuration file rsyncd.conf with your favorite editor. Here you can configure authentication, access, logging and other rsync-daemon modules.

~ vim /usr/local/etc/rsync/rsyncd.conf

We used the following simple configuration for the rsync-daemon.

use chroot = true
hosts allow = 192.168.0.0/24
syslog facility = local5

[Backup]
comment = Encrypted Backup
path = /mnt/./backup
read only = no
list = yes
uid = nobody
gid = nobody

The network-share name is specified in the brackets [Backup]. The location of the encrypted data can be specified by path = /your/backup/location. We added a dot-dir in our path because we use the global use chroot = true option. When this option is used, the rsync daemon will chroot to the path before starting the file transfer with the client system. The dot-dir in the path indicates the point where the chroot should occur. Chrooting to the path has the advantage of extra protection against possible implementation security holes. The hosts allow parameter allows to specify an IP range. If the client IP address does not match the specified range, the connection is rejected.

After saving the edited configuration file, the daemon can be started using the following command (as root):

~ rsync --config=/usr/local/etc/rsyncd.conf --daemon --no-detach

Client

On the client-side we need to create SSH and GPG keys and set up the duplicity tool.

SSH is used to securely authenticate to the remote synchronization server without providing a password. For that, we generated a RSA encrypted key pair for the root user.

~ ssh-keygen -t rsa -b 2048

Press Enter at the password prompt to create a SSH key without a password. The public-key can be transmitted to the synchronization server with the ssh-copy-id tool.

~ ssh-copy-id root@backupserver

After accepting the unverified host, enter the root password of the remote system to transfer your public key.

The GPG keys are generated with the gpg2 OpenPGP encryption and signing tool. gpg2 provides digital encryption and signing services using the OpenPGP standard. With the following command, a key pair using the current default parameters is generated:

~ gpg --gen-key

The dialog shows a series of questions to configure the parameters of the key pair. Construct the user ID from Real Name, Comment and Email Address and enter a strong passphrase to enhance security.

The key-generation will store the keys in a hidden directory at /root/.gnupg/. To get a list of all keys, the gpg keyring can be queried with the following command:

~ gpg --list-keys
/root/.gnupg/pubring.kbx
------------------------
pub   rsa2048/123D442F 2015-07-09
uid       [ultimate] Foo Bar <foo.bar@example.com>
sub   rsa2048/0BE43211 2015-07-09

The list shows that our GPG key id is 123D442F.

System Backup

After the client and server have been prepared, duplicity can be used to create a system backup. duplicity is a tool that incrementally backs data into compressed volumes encrypted with GPG and places them to a storage backend (remote or local).

The duplicity tool has the following syntax to create a backup on a remote storage backend, in our case the rsync-daemon which listens on the server.

~ duplicity --encrypt-key <GPG key ID> --exclude <file or files> <source directory> <target url>

To create a full backup excluding /proc, /sys, /mnt, /tmp, /dev, /var/spool, /var/cache and /var/tmp issue the following command:

~  duplicity --encrypt-key 123D442F --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/tmp --exclude=/dev --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp / rsync://backupserver//mnt/backup
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1436534957.52 (Fri Jul 10 15:29:17 2015)
EndTime 1436547586.24 (Fri Jul 10 18:59:46 2015)
ElapsedTime 12628.73 (3 hours 30 minutes 28.73 seconds)
SourceFiles 132913
SourceFileSize 24120497465 (22.5 GB)
NewFiles 132913
NewFileSize 24120497465 (22.5 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 132913
RawDeltaSize 24073711217 (22.4 GB)
TotalDestinationSizeChange 5053379090 (4.71 GB)
Errors 0
-------------------------------------------------

When the backup is finished duplicity shows some backup statistics.

If you want to force duplicity to perform a full backup even if signatures are available for an incremental backup, just add full after the duplicity command:

~ duplicity full <source directory> <target url>

To force an incremental backup, add incr after the duplicity command:

~ duplicity incr <source directory> <target url>

This will abort if no signatures are available for an incremental backup.

Restore Backup

To restore the backup, just revert the duplicity <source directory> and <target url> parameters, duplicity automatically switches to restore mode. If we wanted to restore all the files we backed up before, we could use this command.

~ duplicity rsync://backupserver//mnt/backup /

It is also possible to restore a specific file or files. The --file-to-restore option causes only <path> to be restored instead of the entire backup. If we wanted to restore the files under /home/foo, we could use this command:

~ duplicity --file-to-restore /home/foo rsync://backupserver//mnt/backup /home/foo

Conclusion

The combination of rsync and duplicity tools provides an easy to use network-capable encrypted backup solution.

Blog