aaronpettigrew.com

You know, when people ask me

“Hey Aaron, how come you can walk through the world with such confidence and grace, always smiling, never ever ever sad?”

or

“Hey Aaron, I’m afraid all the time, but I notice that you aren’t ever scared of anything ever ever ever. What’s the secret to your fearlessness?”

my response is always the same: (cue the booming voice of amazingness)

“Well, friend, I can’t tell you how to become as great as me. But I can help you learn to regularly back up your files, which is the first step to becoming monster-awesomely unafraid and happy.” (end booming voice)

And here’s how you too can become monster-awesomely unafraid and happy:

Use RSync & Cron to do daily backups of your most important files

Today I lent out my usual backup solution (external hard-drive + silverkeeper) to a friend, so I had to figure out a new backup solution to use until he gives it back. I’ve experimented with rolling-my-own backups before (trying to use gmail as a backup server), but without much success. Until now. Today I rolled my very own set of backup scripts, and they’re working great. Here’s how I did it.

Strategizing

First, I thought for a while about what the heck I could back up to. Then it dawned on me: I have an outrageously good hosting plan on aaronpettigrew.com, so I can probably store it all here. Yippee! (For those who’d like to follow along: just modify these instructions to suit whatever backup destination is available. It should work fine.)

Using Rsync

Next, I started playing with rsync, a “[free] open source utility that provides fast incremental file transfer.” (Rsync is a unix app that comes installed on your mac, so no need to download it. Take note, though: if you’re running an OS earlier than 10.4 & you want to preserve resource forks, you might wanna install rsyncx.)

Rsync can be used either on the command line, or through the GUI (located in Applications/Utilities). The GUI is really cool & powerful. Try it out if you’re not interested in playing around with command line stuff. The ‘Quick RsyncX script generator’ is really great — you can set up your backup scheme, sources, destinations (including remote servers!) and some logic for the transfer. Then you can use the ‘Rsyncx scheduler’ to schedule your backups. Neat!

For me, the one thing the GUI lacked was the ability to allow me to easily specify paths to a variety of files/folders that I wanted to transfer all at once. So I went to the command line, where the options are a little more plentiful.

Using Rsync on the Command Line

To get my bearings using Rsync on the command line, I opened the terminal application (located in Applications => Utilities) and typed

man rsync

at the prompt. This brought me to the rsync manual, which is long and boring and really helpful and actually pretty well-written for a unix manual. I scoped out the syntax for using rsync, and decided that, since I was going to backup to a remote machine (but not an rsync server, whatever that is), I’d probably be using something like this:

rsync -[options] SOURCE USER@aaronpettigrew.com:DESTINATION_FOLDER

So, I tried it out, using a small folder of .rtf files as the source, and entering a command much like this one:

rsync -a ~/rsynctest USER@aaronpettigrew.com:rsyncbackups

It worked! Hoorah!

But since it’s not just file transfer I’m looking for, but an incremental backup, I tested a little further: I modified one of the files on my machine locally, then ran exactly the same command a second time. And Voila! When I checked on my server, only the file I’d modified had been re-uploaded; the rest stayed exactly the same. Yay!

Specifying the files/folders to back up

So, now that I had the basics working, I worked on figuring out how to specify only a given set of folders & files to backup. It’s a bit tricky, but I finally got it. Using the

–files-from=FILE

option, I learned that I can create a list of folders and files that Rsync will read for backup. Creating the file is really easy — enter

pico dailybackup_filelist

Then type the paths(without beginning slash) to the files & folders you want to backup, one per line. Hit ctrl x to exit, and type y at the prompt to save. Yay!

Since this is my daily backup (and not my weekly or monthly), I decided that I want to back up my Address Book, my Firefox Bookmarks, my work-related documents, and my calendars from iCal. I thought about adding mail in there too, but then I realized that it’s all backed up on Google’s servers, anyway, so no big deal. So I added the following lines to my file:

Library/Application Support/Firefox
Library/Application Support/AddressBook
Library/Application Support/iCal
Documents/workstuff

Now, here’s where it got tricky. Rsync’s usual archive option (-a), which provides exactly the kind of incremental backup I want, doesn’t recurse directories with the

–files-from=FILE

option, so I had to add the -r in there, so that I get all the contents of the directories I specified. (I also added the -z option, to use compression in transfer & the -v option, for verbosity, so I could see what’s happening during the transfer). The other tricky part is that rsync still requires a SOURCE argument in the command (which took me ages to figure out). It’s hard to explain what I mean, so here’s what my command ended up looking like (note — it’s all on one line):

rsync -razv –files-from=/Users/my_user_name/path_to_dailybackup_filelist /Users/my_user_name/ USER@aaronpettigrew.com:rsyncbackups

Now, I’ll explain:

rsync -razv

- rsync; (-r) search all the files and directories specified; (-a) perform an archival (or incremental) backup; (-z) use compression; and (-v) show me the output.

–files-from=/Users/my_user_name/path_to_dailybackup_filelist

- perform the rsync from the files and directories listed in the file /Users/my_user_name/path_to_dailybackup_filelist

/Users/my_user_name/

- search in the directory /Users/my_user_name/ for all the listed files and folders that I specified before.

USER@aaronpettigrew.com:rsyncbackups

- put my backups on aaronpettigrew.com, in the folder rsyncbackups.

Alright, we’re almost home.

Setting up Rsync through SSH

Before I could pack up this little script, I needed to figure out a way to get Rsync to communicate with my server without passwords, I had to go here to learn about SSH keys. SSH is a secure shell, useful for logging in to remote machines (like servers!). Rsync can use SSH to communicate with servers, which is handy because SSH supports a neat authentication scheme, where I can make a key-file that exists both on my machine and on the server, and SSH will simply match the key-files to authenticate, no need for a password. Neat!

Once I had my ssh key set up, I told Rsync to use SSH to connect to the server by adding

-e “ssh -2″

I tested it out, and Lo! it worked! No passwords!!

So now my whole command looks like this:

rsync -razv -e “ssh -2″ –files-from=/Users/my_user_name/path_to_dailybackup_filelist /Users/my_user_name/ USER@aaronpettigrew.com:rsyncbackups

Cronward!!

Scheduling via cron

Before I could actually set this up in cron, I decided to pack the command I’d made into a nice little script. Why? I dunno, it just seemed tidier. So, I opened pico again and entered

#! /bin/sh

on the first line, to tell the system that this is a script, and then I entered my command on the second line, just as I have it printed above. Then I saved the file as dailybackup.sh, and that was that!

For editing cron, I relied on the awesome O’Reilly tutorial that I found here. These instructions even taught me how to mail myself a message every time the script is run, so that I can know if anything went wrong. Neat! I tested the cron configuration a little before I decided on my final configuration. Here’s how my cronjob looks now:

17 22 * * * sh ~/dailybackup.sh 2>&1 | mail -s “Daily Backup Report” my_user_name

So at 22:17 every day, my computer will automatically backup my important files via dailybackup.sh, and then it’ll mail me a message about how that went. Yay!!

…And that’s how you too can become monster-awesomely unafraid and happy. Good luck!!