Using a storage VPS for backups

Part 5 of the series on choosing the right kind of RackNerd hosting.

Every guide on this site says "keep a backup somewhere else". This one shows you how. A storage VPS is a VPS with lots of disk and modest memory, and it makes a good off-site home for copies of your websites, servers and files. This guide covers how to plan backups, two ways to send them, how to keep them encrypted and how to test that they work.

The examples assume Linux on both ends, using Ubuntu or Debian commands. Tool names are given in plain text: install them with your package manager and follow the project's own documentation for anything beyond the basics. Commands may differ slightly between versions.

What is in this guide

• A few words you will see
• Why a storage VPS suits backups
• Plan before you copy anything
• Before you start
• 1. Prepare the storage VPS
• 2. Give the source a key
• 3. Simple copies with rsync
• 4. Encrypted, versioned backups with restic
• 5. Back up databases properly
• 6. Schedule it
• 7. Test a restore
• Troubleshooting
• Limits
• When to step up
• More in the series

A few words you will see

Term	What it means
Source	The server or computer you are backing up.
Off-site	Stored in a different place from the original, so one problem cannot destroy both.
rsync	A standard tool that copies files and only sends what has changed.
restic, BorgBackup	Backup tools that keep encrypted, versioned backups and store each piece of data only once.
Snapshot	In restic, one saved backup at a point in time. You can restore from any snapshot you keep.
Retention	How many old backups you keep, such as seven daily and four weekly.
Restore	Getting your data back from a backup. The only real test of a backup.

Why a storage VPS suits backups

• Backups need disk, not memory. Copying and storing files uses little RAM. A plan with a lot of disk and 1 GB or 2 GB of memory is usually enough.
• It is off-site. If your main server fails, is hacked or is deleted by mistake, the copy is somewhere else.
• You control it. You choose the tools, the encryption and how long backups are kept.
• It can serve several sources. One storage VPS can hold backups from a few servers and your own computer.

Pick a different location from your main server, so one data centre problem cannot affect both. Choosing a VPS location explains how to compare locations.

Plan before you copy anything

A simple, widely used rule is 3-2-1: keep three copies of important data, on two different kinds of storage, with one of them off-site. A storage VPS is a good off-site copy. It should not be your only one.

Answer these questions first:

Question	Example answer, for illustration only
What needs backing up?	Website files, the database dump, config files in /etc
What can be rebuilt instead?	The operating system and installed packages
How much data is there?	Check with du -sh /var/www
How much can you afford to lose?	One day of changes, so back up daily
How long to keep old copies?	Seven daily, four weekly, six monthly
Who could delete the backups?	Ideally not the source server. See step 2

Leave room to grow. Versioned backups need more space than one copy, though tools like restic only store changed data once.

Before you start

• A storage VPS that you have already secured: updates done, a normal user, SSH keys and a firewall. Your first hour on a new RackNerd VPS walks through this.
• SSH access to the source server.
• A password manager, to keep the backup encryption password safe.

In the commands below, replace STORAGE_IP with the storage VPS address.

1. Prepare the storage VPS

Create a separate user that only holds backups. On the storage VPS:

sudo adduser --disabled-password backup
sudo mkdir -p /home/backup/.ssh
sudo chmod 700 /home/backup/.ssh
sudo chown backup:backup /home/backup/.ssh

This user has no password, so it can only log in with a key, and it has no admin rights.

2. Give the source a key

On the source server, create a key just for backups. Run this as the user that will run the backups:

ssh-keygen -t ed25519 -f ~/.ssh/backup_key -N ""

The key has no passphrase so scheduled jobs can use it. That is why the backup user on the storage VPS has no other rights. Copy the contents of ~/.ssh/backup_key.pub into /home/backup/.ssh/authorized_keys on the storage VPS, then fix the ownership there:

sudo chown backup:backup /home/backup/.ssh/authorized_keys
sudo chmod 600 /home/backup/.ssh/authorized_keys

Test from the source:

ssh -i ~/.ssh/backup_key backup@STORAGE_IP

A stronger setup reverses the direction: the storage VPS logs in to the source and pulls the data. Then an attacker who takes over the source cannot reach the backups. It takes a little more setup, and it is worth considering for anything important.

3. Simple copies with rsync

rsync is the quickest way to start. It keeps one up-to-date copy and only sends changes. On the source:

sudo apt install rsync
rsync -az -e "ssh -i ~/.ssh/backup_key" /var/www/ backup@STORAGE_IP:site/

Install rsync on the storage VPS too. The trailing slash on /var/www/ copies its contents into the site folder.

Know its limits: rsync keeps one copy, not history, and it does not encrypt the files on the storage VPS. If a file is damaged on the source, the damaged version is copied over the good one at the next run. Avoid the --delete option until you understand it, because it also removes files from the backup.

4. Encrypted, versioned backups with restic

For anything that matters, use a backup tool that keeps history and encrypts your data. restic is one common choice and BorgBackup is another. Both are available in Ubuntu and Debian packages. The examples use restic. Follow the project's own documentation for details, as options can change between versions.

On the source, install it and store the repository password in a protected file:

sudo apt install restic
nano ~/.restic-pass
chmod 600 ~/.restic-pass

Put a long random password in that file and save a copy in your password manager. Without it, the backups cannot be read by anyone, including you.

Create the repository on the storage VPS, then make the first backup:

export RESTIC_PASSWORD_FILE=~/.restic-pass
export RESTIC_REPOSITORY=sftp:backup@STORAGE_IP:restic
restic init
restic backup /var/www /etc

restic connects over SSH, so add the backup key to your SSH settings on the source by putting these lines in ~/.ssh/config:

Host STORAGE_IP
  IdentityFile ~/.ssh/backup_key

To keep a set number of old snapshots and free the space used by the rest:

restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

List what you have with restic snapshots, and check the repository now and then with restic check.

5. Back up databases properly

Copying a running database's files can give you a broken backup. Export the database to a file first, then back up that file. For MySQL or MariaDB:

mysqldump --single-transaction --databases mysite > ~/db-backups/mysite.sql

Replace mysite with your database name, and create the db-backups folder first. Keep the database login in a protected options file rather than on the command line, as your database's documentation explains. For PostgreSQL, use pg_dump instead. Then include ~/db-backups in your restic backup.

6. Schedule it

A backup you have to remember is a backup that stops happening. Put the steps in a small script, for example ~/backup.sh:

#!/bin/sh
set -e
export RESTIC_PASSWORD_FILE="$HOME/.restic-pass"
export RESTIC_REPOSITORY="sftp:backup@STORAGE_IP:restic"
mysqldump --single-transaction --databases mysite > "$HOME/db-backups/mysite.sql"
restic backup /var/www /etc "$HOME/db-backups"
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

Make it executable with chmod 700 ~/backup.sh and run it once by hand. If it works, schedule it with crontab -e and a line like this, which runs it at 3 a.m. server time every day:

0 3 * * * $HOME/backup.sh >> $HOME/backup.log 2>&1

Reading /etc may need root. If so, run the script from root's crontab, and keep the key and password files in root's home instead. Check the log after the first scheduled run.

7. Test a restore

A backup you have never restored is a guess. Test one now, and again every few months:

restic snapshots
restic restore latest --target /tmp/restore-test

Look inside /tmp/restore-test and check the files are there and open correctly. For a database, import the dump into a test database. Delete the test folder when you are done. Write down the restore steps somewhere you can reach even if the server is gone.

Troubleshooting

Symptom	Likely cause	Fix
Permission denied (publickey)	Key not in authorized_keys, or wrong ownership	Check the file contents and run the chown and chmod commands in step 2 again
restic asks for a password	RESTIC_PASSWORD_FILE not set in this session	Run the export lines again, or use the script
Cron job does nothing	Cron runs with fewer settings than your shell	Use full paths in the script and check backup.log
Storage VPS disk full	Too many snapshots kept, or prune not run	Run restic forget with --prune, or keep fewer snapshots
Database dump is empty	Login failed or wrong database name	Run the mysqldump command by hand and read the error

Limits

• One backup location is not enough on its own. Follow 3-2-1. Keep another copy elsewhere, such as on your own computer.
• RAID is not a backup. Disk mirroring protects against a failed drive, not against deletion, hacking or mistakes.
• Fair use applies. Unmetered does not mean unlimited. Large daily transfers count towards bandwidth, and very heavy disk use can be limited. Back up changes, not everything, every day.
• Lost passwords mean lost backups. Encrypted backups cannot be recovered without the password.
• Not a file-sharing service. Use the storage VPS for your own backups, within RackNerd's acceptable use policy.
• A storage VPS is still a server. Keep it updated and secured, or it becomes the weak point.

When to step up

If your backups outgrow the disk on a storage plan, you can keep fewer snapshots, back up less, or move to a plan with more disk. For very large amounts of data, a dedicated server with large drives may cost less per gigabyte. For a small amount of data, a pay-per-use cloud storage service may be cheaper and simpler than running a server at all.

To compare what is available now, start with the cheapest RackNerd VPS guide and check the sales page for storage deals.

More in the series

This is part 5 of 5. Previous: What is a Windows VPS for?. Next: back to the start with Shared hosting or a VPS for your first website?. See the whole series.