I just found a great little utility to test backup systems.
It’s called Backup Bouncer and like the bouncer in a bar Backup Bouncer is there to keep the patrons honest. It will let you know ahead of time if your backup system is letting you down in complex ways, like not copying metadata or is blowing out resource forks or resetting creation dates.
These are the sorts of things you won’t notice until you’ve lost your original and for some reason your Aperture or iPhoto library won’t run properly anymore.
Of course, even a defective backup is better than none.
I was happy to see that my main backup tool SuperDuper! passes the test with flying colours. (For full bootable backup, Mike Bombich’s CCC (Carbon Copy Cloner) is back in the game as well, after a couple of difficult years, passing all tests as well – as a past donater, I guess I own CCC as well.)
That’s great news as it means I don’t have to test SuperDuper! myself . Indirectly I do test SuperDuper! by booting from my bootable backup after most backups and doing a bit of work just to be sure that the bootable backup really boots and really works.
But as good as SuperDuper! is for a whole drive bootable backup, is it (and CCC) awkward for backing up a directory or two. You need a second program to be moving image or music files back and forth between two computers. And this second sync program is a bigger problem.
My secondary tool is for syncing directories and moving anything from 500 MB to 50 GB of data around. I use a little application called MimMac which is very easy to use and inexpensive ($10/per computer).
But MimMac is a bit of a black box. We don’t really know what goes on inside. Everything seems fine, but what exactly is MimMac copying and how well?
As the backups are not bootable, MimMac is more difficult to stress test.*
Here’s the Backup Bouncer report to save you the trouble of setting it all up and running it yourself:
Verifying: basic-permissions ... ok (Critical) Verifying: timestamps ... ok (Critical) Verifying: symlinks ... ok (Critical) Verifying: symlink-ownership ... ok Verifying: hardlinks ... FAIL (Important) Verifying: resource-forks ... Sub-test: on files ... ok (Critical) Sub-test: on hardlinked files ... FAIL (Important) Verifying: finder-flags ... ok (Critical) Verifying: finder-locks ... ok Verifying: creation-date ... ok Verifying: bsd-flags ... ok Verifying: extended-attrs ... Sub-test: on files ... ok (Important) Sub-test: on directories ... ok (Important) Sub-test: on symlinks ... ok Verifying: access-control-lists ... Sub-test: on files ... ok (Important) Sub-test: on dirs ... ok (Important) Verifying: fifo ... FAIL Verifying: devices ... ok Verifying: combo-tests ... Sub-test: xattrs + rsrc forks ... ok Sub-test: lots of metadata ... ok
Not bad. The failures in hardlinks and resource-files in hardlinked is similar to Apple’s cp-copy command. Failure in fifo only happens in ditto. FIFO stands for first-in-first-out. As far as I can tell, FIFO is not mission-critical for personal/local backups.
What MimMac gets right that most of the other methods do not get right is metadata. So Benjamin is paying attention. Still not clear what copying engine he’s using though.
What is good about MimMac is that it is very fast. What is not so good about MimMac is that you can’t do a test run. Either you run your sync or you don’t so you can’t find out about conflicts or mistakes before you press go. The speed probably makes up for the risk.
I would like to recommend MimMac but can’t due to licensing methods. MimMac relies on the esellerate engine for license verification and each license is tied to your specific hardware. If Benjamin goes out of business or just gets tired of MimMac and you upgrade your computer or your hard drive, you have no further access to the software. Full stop. Period. For core programs, I much prefer either open-source (commercial open-source is fine, it doesn’t have to be GNU) or if not open-source, at least just a personal license code which will continue to work even if the developer decides to stop work. I’ve lost enough software over the years to developers leaving the business, that there is no way I want my core functions dependent on whether another person’s whim or even health.
Moreover when you switch computers, all software which is tied to hardware either has to be unlicensed and relicensed (forget it!) or it requires emails and phone calls to the developer (one obnoxious developer once told me for his $25 utility as a courtesy he would allow me to license it on my new computer once, but next time I had to unlicense his utility or he wouldn’t issue a replacement key – what do these developers think: their two-bit utility is one of five applications we own: this licensing system just doesn’t scale and reminds me of the Lubyanka in Moscow).
The worst developer in Mac backup actually runs background spyware applications on your network full-time if you decide that you want to use his software. The problem is that the spyware not only spies but steals significant background cycles and is constantly pinging the inside of your network, creating no end of dead-end traffic. While the solution is speedy, slowing down my computer is not on. At this point, licensing this guy’s software is extremely dicey: you need to give him special codes for your hardware (not even the standard ones) and if you’re lucky it might just work. Talk about shooting yourself in the foot. He’s made using his software so painful that from being the best solution, he’s become the worst solution – as well as the most expensive. I guess he’s been taking lessons from the RIAA paranoid and self-destructive school of copyright.
So I am looking for an rsync GUI, paid or not. One rsync GUI can always be replaced with another in the worst case. Unfortunately rsync doesn’t pass the Backup Bouncer test unless you do a special install which means you have to tinker on all your computers (slowing you down) and that a certain number of these GUI won’t work as they will be defaulting to the built-in Apple rsync.
Here’s a couple of candidates to save you the time of searching for rsync GUIs (there’s a lot of abandonware out there):
- aRsync 0.41. I don’t like that a simple rsync GUI is 4.7 MB – what are they hiding in there? I don’t like the Pirate logo on backup software. This is the kind of software which could compromise your whole hard drive or send out your financial data. No to pirates or unknown entities. I don’t like betas for backup or sync software either. This is an area where you need 100% reliablility. Moreover aRsync fails many tests including semantic links, hard links, creation date, fifo and metadata. Ouch.
- Simple Sync 1.1. I don’t know how well SimpleSync works. Perhaps very well if you do follow the instructions for updating rsync. If not certainly it will work as well as Apple’s rsync with. It’s worth noting that Simple Sync is just 210 KB – that’s about right for a wrapper – and that Kevin includes both his mobile number and a link to his main company’s home page on the Simple Sync page. I’m feeling much better about using this script already.
Maybe we will build a self-contained advanced rsync GUI ourselves and release it so that we can get the right version of rsync and a GUI – and then we can share it with the world. For the moment, SuperDuper! and MimMac are keeping us safely backed up and synced.
Whatever you do, don’t forget to backup!
At least once a week.
For those interested in specific backup strategies for Mac OS X for photographers and other media intensive users, I’ve written another article called The Backup Manifesto.
* MimMac is capable of bootable backups but I have more trust in SuperDuper! both for technical reasons and for licensing reasons as outline later in the article. On the other hand, if you are willing to accept MimMac’s licensing you can probably take a pass on SuperDuper! and use MimMac for everything, saving yourself $28 to spend on a replacement whenever Benjamin decides to abandon MimMac or change his licensing and MimMac won’t run on your computer anymore.
Alec Kinnear
Alec has been helping businesses succeed online since 2000. Alec is an SEM expert with a background in advertising, as a former Head of Television for Grey Moscow and Senior Television Producer for Bates, Saatchi and Saatchi Russia.
I agree with you about SuperDuper. Great for full backups but not that great for specific directories… Have you checked out Synk? decimus.net/synk_standard.php It seems several universities use this & recommend it to their students and staff–because it’s simple & cheap. I have yet to try it out and confirm using Backup Bouncer. (I’ve installed the most recent rsync and successfully done some backups but I hate spending time tinkering with my scripts to get them just so…)
Hi Woo,
I think I own a copy of Synk and I hate it. It is slow, clunky and unintuitive.
For big backups I used SuperDuper!. I also keep a TimeMachine backup (software is free, quite cool and only costs you one more hard drive to use). For directories, I am using MimMac.
I’m curious about that “worst developer” you mention that installs spyware. You keep him anonymous. Can you give a hint? Like: does it happen to be a developer from, let’s say, Germany?
Actually the backup developer who installs spyware and a permanent processor cycle stealing network monitor is an American.
But why don’t you mention the bad German developer? It would be good to get these bad developers names out in the open.
Alec,
As far as I know there is no need to put the German (actually Croat) developer into the negative spotlight. I met him at a workshop he organised and he seems to be a friendly guy who is dedicated to his software/customers.
I still need to thoroughly test his software, but your price remark made me think of him because his company targets professional users willing to pay a premium price (but still acceptable for small business) for his software that has a lot of features. So far it gets positive reviews.
The software is “PresSTORE”, downloadable from http://www.archiware.de (30 day free trial). It may have a steep learning curve.
I’m too busy with other things to do the testing right now -will do that later this year- but if someone else has a few free hours available on a rainy sunday afternoon… You may need a tape drive to be able to do a full test.
Threads like this one (and in other places) show in my opinion that the Mac OS X backup market could be more competetive, although things seem to move in that direction. I don’t think Time Machine is a serious option except for home users. The way it deletes old backup folders makes it not very reliable in my opinion, also TM relies on file system events and who knows how reliable these are?
A simple program to make backups from folders that I’ve used in the past is Synchronize Plus. It was an OK program, except that it starts an annoying ‘phone home’ background process every time you start it.
Hi Bastiaan,
Thanks for sharing.
The Croat developer Archiware has nothing to do with reasonable Mac backup solutions. He is attempting to do cross-platform industrial level backup. His low end product Backup2Go comes in at €500. The guy doesn’t even have screenshots up. I’m sure the program is Medusa ugly as cross-platform solutions tend to be. Not impressed.
What’s interesting is that the spyware you name, Synchonize Plus is from the developer who could have owned the Mac backup market if he weren’t such a paranoid disagreeable chap. Instead of serving customers and improving his software, he’s spent most of the last five years on anti-piracy schemes (end result of his anti-piracy schemes was that I was unable to use the software as a licensed user while I could have used the software with a cracked copy).
Insanity.
I stand by my guns: weekly/monthly SuperDuper! backups are the main solution. For folder synchronisations we are underserved, with either underpowered or slightly impenetrable interfaces like Mimmac or the much worse Synk. LaCie’s SilverKeeper will also do if you manage to get a copy of it (SliverKeeper is marketingware (you have to sign up for newsletters to get it). Apparently LaCie has already made SilverKeeper Snow Leopard compatible (there are advantages to corporate backing sometimes).
F.y.i. The LaCie newsletters are optional! Just uncheck the boxes next to them and download your copy of SilverKeeper
You may be interested in the list of rsync GUI’s listed on the LBackup website : lbackup.org/about#alternatives_to_lbackup
I use SD! all the time to copy folder to folder. There is a command-line binary inside the SD! application bundle, “Contents/MacOS/SDCopy”, which can be observed running when one launches the SD! application, and whose command-line arguments are easily reverse-engineered.
Do not even think to ask them for support using SDCopy. Anyone who needs to ask for help using SDCopy shouldn’t be using it.
Is this better than the latest rsync? No. Is it convenient, if you own SD!? Yes.
Hi John,
That’s a superb tip. Any chance you could just share the basics with us here. I’ve just been using ditto for rescuing a hard drive gone sour and would have loved to have used the SD! command line functions instead.
Thanks!
Start the paid version of SD! on a “Smart Update” copy from the GUI. While it runs, go to Terminal and type
ps -ax | egrep S[D]Copy
You’ll extract the exact call to SDCopy, something like
/Applications/SuperDuper!.app/Contents/MacOS/SDCopy -v -v -p -c different -x -a copy -d /Users/dave/Library/Application Support/SuperDuper!/Copy Scripts/Standard Scripts/Backup – all files.dset -u “John Fitzgerald” -k 9999999999999999999 /Volumes/User /Volumes/bUser
Any arguments with spaces, such as my name or the folder locations, WON’T be quoted but need to be quoted. This is a basic Unix issue; if one doesn’t recognize this as a trivial problem, there are other gotchas one will probably encounter.
Substitute your license code, as it is shown when your copy of SD! runs.
This command line program is only helpful if one has a clear idea what SD! already does before calling it. For instance, that “Apple thinks we’re dumb as stumps!!!” default of not enabling permissions on newly mounted drives. Without enabling permissions (“sudo vsdbutil -i” will fix everything currently mounted, but don’t try it without understanding it) SDCopy will just make a mess.
That said, SDCopy happily copies folder to folder, including remote targets like those mounted by ExpanDrive. It is what it is; if your remote copy doesn’t boot or have a faithful Time Machine backup, it is entirely unreasonable to expect SD! to hold your hand. Knowing SDCopy just saves you time if you could have written it yourself, and you know what you’re doing.
Read everything you can at http://www.bombich.com to get a better idea what CCC and SD! do. The site is hard to navigate, e.g. try finding the must-read bombich.com/mactips/rsync.html from the site index. In particular, he tells you exactly what files not to copy for a bootable system, somewhere on that site and again in plain text inside CCC. He probably mentions “blessing” a system folder too, but I couldn’t find it there; I already knew.
Great John. I was able to grab my paths while running SD! – ps -ax | egrep S[D]Copy worked perfectly.
I think the GUI to SuperDuper! is superb for what it is that it does which is backing up whole volumes.
Why would using the SuperDuper! command be better than ditto for backing up folders?
Some ideal usage scenarios with the exact options would be a godsend.
Huh? I don’t know. I thought that you were the one that wanted to use SD! Now you’ve got me really confused. Have I been a complete idiot going to all that trouble using SD! when I could have been using ditto?
Why don’t you make two folders “source” and “target” and copy the same 1 GB+ movie into each of them. Then try the following commands in turn, and report back on the timings:
time rsync source target time ditto source target time SDCopy source target
where SDCopy is an alias to SDCopy with the options set to Smart Update.
In general, everyone wants different options, but if you set the options you want in the SD! GUI, then inspect what gets passed to SDCopy, you know what to do.
You’re giving me insight into why SD! wants absolutely nothing to do with supporting this command line tool. PLEASE run considered experiments to answer your own questions, before asking me to write a blog page in your comments.
Woa John, chill.
Do you have any idea how many projects we have on the go at Foliovision right now? There’s no way I have time to do experiments on command line tools. I could run the tool if I had the documentation but don’t have time to create the documentation for myself. It seemed you had already reverse engineered the SD! tool so you could share your cheat sheet with us.
Like I share my tests on OS X software (go have a look at yesterday’s Little Snitch and hosts article).
Thanks for letting us know about the tool.
If you are scheduling command line backup of a folder on your system, LBackup is an open source rsync based backup tool which you may want to consider.
[this is a promotional comment from the developer of lbackup linked to his commercial backup service. on the other hand lbackup does look like an interesting interface to rsync. unfortunately there are no screenshots. there is zero feedback at macupdate. you are really on your own with this one: pursue at your own risk. alec]