Backup on Mac OS X: Testing MimMac with Backup Bouncer

Sunday, January 18th, 2009

I just found a great little utility to test backup systems.

It's called Backup Bouncer and like the bouncer in a bar Backup Bouncer is there to keep the patrons honest. It will let you know ahead of time if your backup system is letting you down in complex ways, like not copying metadata or is blowing out resource forks or resetting creation dates.

These are the sorts of things you won't notice until you've lost your original and for some reason your Aperture or iPhoto library won't run properly anymore.

Of course, even a defective backup is better than none.

I was happy to see that my main backup tool SuperDuper! passes the test with flying colours. (For full bootable backup, Mike Bombich's CCC (Carbon Copy Cloner) is back in the game as well, after a couple of difficult years, passing all tests as well - as a past donater, I guess I own CCC as well.)

That's great news as it means I don't have to test SuperDuper! myself . Indirectly I do test SuperDuper! by booting from my bootable backup after most backups and doing a bit of work just to be sure that the bootable backup really boots and really works.

But as good as SuperDuper! is for a whole drive bootable backup, is it (and CCC) awkward for backing up a directory or two. You need a second program to be moving image or music files back and forth between two computers. And this second sync program is a bigger problem.

My secondary tool is for syncing directories and moving anything from 500 MB to 50 GB of data around. I use a little application called MimMac which is very easy to use and inexpensive ($10/per computer).

But MimMac is a bit of a black box. We don't really know what goes on inside. Everything seems fine, but what exactly is MimMac copying and how well?

As the backups are not bootable, MimMac is more difficult to stress test.*

Here's the Backup Bouncer report to save you the trouble of setting it all up and running it yourself:

Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok 
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok 
Verifying:        creation-date ... ok 
Verifying:            bsd-flags ... ok 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... FAIL 
Verifying:              devices ... ok 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 

Not bad. The failures in hardlinks and resource-files in hardlinked is similar to Apple's cp-copy command. Failure in fifo only happens in ditto. FIFO stands for first-in-first-out. As far as I can tell, FIFO is not mission-critical for personal/local backups.

What MimMac gets right that most of the other methods do not get right is metadata. So Benjamin is paying attention. Still not clear what copying engine he's using though.

What is good about MimMac is that it is very fast. What is not so good about MimMac is that you can't do a test run. Either you run your sync or you don't so you can't find out about conflicts or mistakes before you press go. The speed probably makes up for the risk.

I would like to recommend MimMac but can't due to licensing methods. MimMac relies on the esellerate engine for license verification and each license is tied to your specific hardware. If Benjamin goes out of business or just gets tired of MimMac and you upgrade your computer or your hard drive, you have no further access to the software. Full stop. Period. For core programs, I much prefer either open-source (commercial open-source is fine, it doesn't have to be GNU) or if not open-source, at least just a personal license code which will continue to work even if the developer decides to stop work. I've lost enough software over the years to developers leaving the business, that there is no way I want my core functions dependent on whether another person's whim or even health.

Moreover when you switch computers, all software which is tied to hardware either has to be unlicensed and relicensed (forget it!) or it requires emails and phone calls to the developer (one obnoxious developer once told me for his $25 utility as a courtesy he would allow me to license it on my new computer once, but next time I had to unlicense his utility or he wouldn't issue a replacement key - what do these developers think: their two-bit utility is one of five applications we own: this licensing system just doesn't scale and reminds me of the Lubyanka in Moscow).

The worst developer in Mac backup actually runs background spyware applications on your network full-time if you decide that you want to use his software. The problem is that the spyware not only spies but steals significant background cycles and is constantly pinging the inside of your network, creating no end of dead-end traffic. While the solution is speedy, slowing down my computer is not on. At this point, licensing this guy's software is extremely dicey: you need to give him special codes for your hardware (not even the standard ones) and if you're lucky it might just work. Talk about shooting yourself in the foot. He's made using his software so painful that from being the best solution, he's become the worst solution - as well as the most expensive. I guess he's been taking lessons from the RIAA paranoid and self-destructive school of copyright.

So I am looking for an rsync GUI, paid or not. One rsync GUI can always be replaced with another in the worst case. Unfortunately rsync doesn't pass the Backup Bouncer test unless you do a special install which means you have to tinker on all your computers (slowing you down) and that a certain number of these GUI won't work as they will be defaulting to the built-in Apple rsync.

Here's a couple of candidates to save you the time of searching for rsync GUIs (there's a lot of abandonware out there):

  • aRsync 0.41. I don't like that a simple rsync GUI is 4.7 MB - what are they hiding in there? I don't like the Pirate logo on backup software. This is the kind of software which could compromise your whole hard drive or send out your financial data. No to pirates or unknown entities. I don't like betas for backup or sync software either. This is an area where you need 100% reliablility. Moreover aRsync fails many tests including semantic links, hard links, creation date, fifo and metadata. Ouch.
  • Simple Sync 1.1. I don't know how well SimpleSync works. Perhaps very well if you do follow the instructions for updating rsync. If not certainly it will work as well as Apple's rsync with. It's worth noting that Simple Sync is just 210 KB - that's about right for a wrapper - and that Kevin includes both his mobile number and a link to his main company's home page on the Simple Sync page. I'm feeling much better about using this script already.

Maybe we will build a self-contained advanced rsync GUI ourselves and release it so that we can get the right version of rsync and a GUI - and then we can share it with the world. For the moment, SuperDuper! and MimMac are keeping us safely backed up and synced.

Whatever you do, don't forget to backup!

At least once a week.


For those interested in specific backup strategies for Mac OS X for photographers and other media intensive users, I've written another article called The Backup Manifesto.

* MimMac is capable of bootable backups but I have more trust in SuperDuper! both for technical reasons and for licensing reasons as outline later in the article. On the other hand, if you are willing to accept MimMac's licensing you can probably take a pass on SuperDuper! and use MimMac for everything, saving yourself $28 to spend on a replacement whenever Benjamin decides to abandon MimMac or change his licensing and MimMac won't run on your computer anymore.

WordPress | 15 comments

The Backup Manifesto

Tuesday, October 14th, 2008

I don’t know how many hard drives you have but as a photographer and filmmaker I have over a dozen hard drives, scattered across three computers.

It used to be enough just to pair up some hard drives and move a few files around.

No more. Each computer needs its own backup drive/system.

I’ve just been through cleaning up duplicate backups and freeing up about five drives.

While doing so, I had to come up with some principles of backup, which I will call the backup manifesto. Here they are:

  • more backup is better
  • all drives to be named and labeled at all times (how many Seagate 200 GB drives – first near silent hard drives available – can one photographer have? about half a dozen)
  • dispose of dead drives immediately (you have to identify them)
  • working drives go into USB cases of their own
  • when in doubt, don’t erase but retire.
  • you need backup of your backup. Always pair drives.
  • less working drives is easier to manage i.e. 2×1000 GB is better than 10×200 GB
  • back up 5×200 GB to 1×1000 GB. This liberates 5×200 GB (second half of the pairs) and leaves you with a single working drive with access to all your archived material.
  • bare SATA drives with a SATA external plug is space/power supply efficient.
  • drives are cheap – even good ones. Buy top of the line well-tested drives in the largest size readily available (usually one increment down from the current maximum). Find the extra $50. Your time and nerves are worth the expense.
  • get a good disk cataloging program. I use CDfinder. You don’t want to have to handplug six ATA drives every time you need to find a single old file.
  • if you are serious about using your disks efficiently, get a good disk space calculator (I use Baseline) which will also tell you about duplicate files. Using Baseline, I was able to clean out 30 GB of duplicate files on my Macbook boot drive in half an hour. It would have taken days to do this work by hand with just the finder.
  • keep your boot drives only 70% full (maximum!). The last 30% of your hard drive is the slowest part. You don’t want to be booting from the slowest part – your new $2000 laptop will be running at about 60% efficiency. (Different rules apply to RAID arrays but even there you’d want to keep 10 or 15% clean to make sure you don’t crash it by filling all the free space.)
  • upgrade your backups to new media when you can put 4 or 5 of them on the new large standard.
  • firewire 400 is more expensive and somewhat unreliable. firewire 800 is still more expensive and very unreliable. Use firewire sparingly, only for fast media drives.
  • if you insist on snapshots of old boot drives, just make them disk images on the big drives. You don’t really need those snapshots. Trust me – you won’t be going back to Windows 98 or OS X 10.1.5 or 10.2.3 (you might just go back to OS 8/9 for a legacy program or two so there are exceptions). Personally I haven’t. Mac OS X is getting a lot better and so are some of the new programmers. User interface is enough better that using vintage software is really a chore. Running any kind of virtualisation puts a big burden on your computer’s resources. The less of it you can do the better (I don’t even run Rosetta 99% of the time, let alone Classic).

When you are done, you should end up with two workhorse 750/1000 GB hard drives with a bunch of retired backups and a handful of free 160 GB to 300 GB hard drives available for various purposes (scratch drives/backup drives for friends).

You should almost never have to go to the retired backups to pull a file. Put them somewhere safe with a stable temperature. If they have sensitive information on them (no reason they need to – your sensitive information can be kept on one live drive and its two current backups), buy yourself an inexpensive safe and keep the retired drives there.

The retired drives won’t be used except in emergency so they should have a lifespan of another ten years. If you do have to pull a retired drive out due to a workhorse drive going down, you should duplicate it out to new media immediately.

Update - 18 January 2009: I've added an article with specific OS X back up software recommendations including a Bouncer Backup stress test on MimMac.

IT | 5 comments