Brett Klamer

Backups with Tarsnap

Spideroak removed their warrant canary in early August 2018. Whether it was through pure ignorance, or malfeasance, they’ve sent a clear signal to drop their service. One of the few, if not better, alternatives is https://www.tarsnap.com.

Install Tarsnap

See the documentation at https://www.tarsnap.com/documentation.html. The only modification to the /etc/tarsnap.conf file I made was to allow humanize-numbers.

Automatic Backups

The easiest way to get started is a simple cronjob. Run sudo crontab -e and insert:

# tarsnap backup every 2nd hour (even hours, i.e. 10, 12, 2, etc.) of the day.
0 */2 * * * /root/tarsnap-backup.sh

where the script below is created at /root/tarsnap-backup.sh. Don’t forget to chmod +x /root/tarsnap-backup.sh

#!/bin/bash

BASELINE_SIZE=16000
# Make this an absolute path
BACKUP_DIR='/path/to/backup/'
CHECK_FILE='/tmp/tarsnap-check.txt'
BACKUP_NAME='NAME'

# Save dry run stats
tarsnap -c -f dry_run --dry-run --print-stats ${BACKUP_DIR} 2> ${CHECK_FILE}

# Extract and edit size of new data that will be saved
## Get the 3rd and 4th columns of the 6th row. This works with the standard 
## print output with the `humanize-numbers` option.
ACTUAL_SIZE=`awk '{ print $3,$4 }' /tmp/tarsnap-check.txt | sed '6q;d'`
## Remove the last character in the string
ACTUAL_SIZE=${ACTUAL_SIZE::-1}
## Remove any spaces
ACTUAL_SIZE=${ACTUAL_SIZE//[[:blank:]]/}
## Make sure letter is capitalized
ACTUAL_SIZE=${ACTUAL_SIZE^^}
## Convert units to bytes
ACTUAL_SIZE=`numfmt --from=auto ${ACTUAL_SIZE}`

# If new data is more than baseline, run a backup
if (("$ACTUAL_SIZE" > "$BASELINE_SIZE")); then
    /usr/bin/tarsnap -c \
        -f "${BACKUP_NAME}-$(date +%Y-%m-%d_%H-%M-%S)" \
        --print-stats \
        ${BACKUP_DIR}
fi

# remove temporary check file
rm -rf ${CHECK_FILE}

This script checks if there is new data to backup and, if so, will create a backup every two hours. This can leave a lot of old archives hanging around, so you may want to check out a few unofficial helper scripts that provide structured archive creation and deletion. The best of which look to be https://github.com/miracle2k/tarsnapper and https://github.com/alexjurkiewicz/acts.

Otherwise you can just delete everything older than a chosen date by looking at

tarsnap --list-archives | sort > "./tarsnap-delete.sh"

and modifying the file into a deletion script.

Recovering files

Restore an archive using:

# Definitions of tarsnap options
# `-x` Extract files to disk
# `-t` List archive contents to stdout
# `-v` Verbose, will list each file name during extraction
# `-f` Operations will be made on specified archive name
# `-O` Files will be written to stdout
# `-C` Change directory
# `>` Redirect stdout to specified file

# Restore whole archive directory in place (will overwrite current files)
tarsnap -xvf backupname
# Restore whole archive directory to new location (no overwriting)
tarsnap -xvf backupname -C ./restore/path
# Restore specific directory to new location (no overwriting)
tarsnap -xvf backupname file/path/to/restore/* ./restored/path
# Restore single file to new location (no overwriting)
tarsnap -O -xvf backupname file/path/to/restore.txt > ./restored-file.txt
# Search for location of file in backup and then list matches
tarsnap -tf backupname | grep keyword

Reinstalling Tarsnap

After reinstalling an operating system or moving to a new machine:

  1. Install tarsnap as usual.
  2. Move the old key to /root/tarsnap.key.
  3. If you have a large backup, you may want to copy over the latest version of the old cache directory located at /usr/local/tarsnap-cache. Otherwise run sudo tarsnap --fsck-prune to rebuild the local cache.

Tarsnap GUI

There is an official GUI for tarsnap at https://github.com/Tarsnap/tarsnap-gui. It provides basic interaction with the command line utility. The cron scheduler is hit-or-miss. The archive window is nice for sorting through and deleting old archives.

Last updated 2020-09-08