A backup script you've tested
Automated, off-site, and restored at least once.
Build time ~3 hrs
Backups are simple to write and easy to get subtly wrong in ways you only discover when you need them. The restore is the part that matters, and it's the part nobody tests.
1. Write both halves at once
Never write a backup script without writing the restore script in the same session. If you can't describe how to restore it, you haven't finished.
Write a backup script for <database / files / both>. Host: <host>. - Dump/copy consistently — a database dump must be a coherent snapshot, not a file read while writes are happening. - Compress, and encrypt before it leaves the machine. Key from an environment variable. - Upload to storage on a DIFFERENT provider from where the data lives. - Retention: keep <daily for n, weekly for n, monthly for n>, and actually delete beyond that. - Verify the uploaded file: size, and that it can be read back and decompressed. A backup that failed silently is the worst outcome. - Exit non-zero on any failure, and notify me either way. Then write the restore script, and the exact steps to test it into a scratch environment.
2. Notify on success, not just failure
Counterintuitive but important. If you only hear about failures, silence is ambiguous — it means either "fine" or "the script stopped running a month ago".
A daily one-line "backup completed, 412MB" confirms the whole chain is alive.
3. Off-site means a different provider
A backup in the same account as the production database is not off-site. It protects against your mistake; it doesn't protect against the account being lost, compromised, or suspended.
4. Encrypt before upload
Your backup contains everything — user records, credentials, the lot. Encrypt it on your machine, with a key stored somewhere separate from the backup. And store that key somewhere you can reach when your infrastructure is down.
5. Then actually restore it
Today. Into a scratch environment. Time it, and write down how long it took.
That number is what you'll need when someone asks "how long until we're back?", and it's routinely three times longer than people assume.
Put a restore test in your calendar quarterly. Backup chains break silently, and the failure only surfaces when you need them.