Backups with Tarsnap

Spideroak removed their warrant canary in early August 2018. Whether it was through pure ignorance, or malfeasance, they’ve sent a clear signal to drop their service. One of the few, if not better, alternatives is https://www.tarsnap.com.

Install Tarsnap

See the documentation at https://www.tarsnap.com/documentation.html. The only modification to the /etc/tarsnap.conf file I made was to allow humanize-numbers.

Automatic Backups

The easiest way to get started is a simple cronjob. Run sudo crontab -e and insert:

# tarsnap backup every 2nd hour (even hours, i.e. 10, 12, 2, etc.) of the day.
0 */2 * * * /root/tarsnap-backup.sh

where the script below is created at /root/tarsnap-backup.sh. Don’t forget to chmod +x /root/tarsnap-backup.sh

#!/bin/bash

BASELINE_SIZE=16000
# Make this an absolute path
BACKUP_DIR='/path/to/backup/'
CHECK_FILE='/tmp/tarsnap-check.txt'
BACKUP_NAME='NAME'

# Save dry run stats
tarsnap -c -f dry_run --dry-run --print-stats ${BACKUP_DIR} 2> ${CHECK_FILE}

# Extract and edit size of new data that will be saved
## Get the 3rd and 4th columns of the 6th row. This works with the standard 
## print output with the `humanize-numbers` option.
ACTUAL_SIZE=`awk '{ print $3,$4 }' /tmp/tarsnap-check.txt | sed '6q;d'`
## Remove the last character in the string
ACTUAL_SIZE=${ACTUAL_SIZE::-1}
## Remove any spaces
ACTUAL_SIZE=${ACTUAL_SIZE//[[:blank:]]/}
## Make sure letter is capitalized
ACTUAL_SIZE=${ACTUAL_SIZE^^}
## Convert units to bytes
ACTUAL_SIZE=`numfmt --from=auto ${ACTUAL_SIZE}`

# If new data is more than baseline, run a backup
if (("$ACTUAL_SIZE" > "$BASELINE_SIZE")); then
    /usr/bin/tarsnap -c \
        -f "${BACKUP_NAME}-$(date +%Y-%m-%d_%H-%M-%S)" \
        --print-stats \
        ${BACKUP_DIR}
fi

# remove temporary check file
rm -rf ${CHECK_FILE}

This script checks if there is new data to backup and, if so, will create a backup every two hours. This can leave a lot of old archives hanging around, so you may want to check out a few unofficial helper scripts that provide structured archive creation and deletion. The best of which look to be https://github.com/miracle2k/tarsnapper and https://github.com/alexjurkiewicz/acts.

Otherwise you can just delete everything older than a chosen date by looking at

tarsnap --list-archives | sort > "./tarsnap-delete.sh"

and modifying the file into a deletion script.

Excluding files

An easy way to exclude files from backups is to set the nodump flag using

# set flag
chattr +d no/backup/dir
# remove flag
chattr -d backup/dir
# verify attribute is set
lsattr

Alternatively, you can add --exclude no/backup/dir to the tarsnap command (must be inserted before the backup path)

Recovering files

Restore an archive using:

# Definitions of tarsnap options
# `-x` Extract files to disk
# `-t` List archive contents to stdout
# `-v` Verbose, will list each file name during extraction
# `-f` Operations will be made on specified archive name
# `-O` Files will be written to stdout
# `-C` Change directory
# `>` Redirect stdout to specified file

# Restore whole archive directory in place (will overwrite current files)
tarsnap -xvf backupname
# Restore whole archive directory to new location (no overwriting)
tarsnap -xvf backupname -C ./restore/path
# Restore specific directory to new location (no overwriting)
tarsnap -xvf backupname file/path/to/restore/* ./restored/path
# Restore single file to new location (no overwriting)
tarsnap -O -xvf backupname file/path/to/restore.txt > ./restored-file.txt
# Search for location of file in backup and then list matches
tarsnap -tf backupname | grep keyword

Reinstalling Tarsnap

If you reinstall an operating system, or replace an old computer, you may want to continue backups using the same key and machine name. It should be as simple as:

1. Install tarsnap as usual.
2. Move the old key to /root/tarsnap.key.
3. If you have a large backup, you may want to copy over the latest version of the old cache directory located at /usr/local/tarsnap-cache. Otherwise run sudo tarsnap --fsck-prune to rebuild the local cache.

However, if you have two or more computers uploading to your tarsnap account, you will want them to have separate keys and machine names. For more information, see here http://mail.tarsnap.com/tarsnap-users/msg00768.html

To create a key and register a new computer in one step, follow something like:

tarsnap-keygen --keyfile /root/tarsnap.key --user USERNAME --machine MACHINENAME

Tarsnap GUI

There is an official GUI for tarsnap at https://github.com/Tarsnap/tarsnap-gui. It provides basic interaction with the command line utility. The cron scheduler is hit-or-miss. The archive window is nice for sorting through and deleting old archives.