One of our clients had his weblog cut off with the dreaded Bandwidth Limit Exceeded notice:
Bandwidth Limit Exceeded
The server is temporarily unable to service your request due to the site owner reaching his/her bandwidth limit. Please try again later.
Richard’s visitors had pumped 80 GB out already this month on his FreeTheAnimal weblog primal living weblog. Not bad for a single writer not in a formal network. Every month his traffic is growing. Congratulations Richard!
These 80 GB of bandwidth are the real thing, with just a few slightly overweight images, not a single big file accidentally uploaded.
Richard was surprised and upset to see his weblog cut off as were we. While most hosts cut clients off as a routine matter of business, we do not. We treat our clients as we would like to be treated ourselves.
Two of Foliovision’s core hosting policies are:
never cut off a client’s site for bandwidth
no bandwidth overages
Clearly this is a clear violation of our hosting and customer service commitment. We recently moved from the dreadful hSphere control panel to the comparative friendliness of cPanel. We knew how to fix this issue in hSphere but had missed adding it in our default cPanel server setup. Here's how you make cPanel behave much more courteously with your customers.
Start by turning off bandwidth cutoffs. It’s in WHM under Server Configuration, Tweak Settings:
whm tweak settings bandwidth limits
A very big page of options come up. Search for: “Disable Suspending accounts that exceed their bandwidth limit”:
disable suspending accounts bandwidth limit
While you are at it, you may as well hit up your users with email warnings at 80 and 90 and 95% bandwidth so they can ask for an upgrade or at least are ready for their upgrade when it comes. Unfortunately, these notifications only go to the client and not to you.
To go without bandwidth overage cutoff this, we need to know when clients are exceeding bandwidth so we can upgrade them. There doesn’t seem to be any simple notification system for admins. A workaround would be to create an account notification email for each client on your own hosting email account which is forwarded to them and then to you. Advantage is that you’d see what snarky emails your automated server software is sending out on your behalf to alienate clients. Still that’s lots of extra work when setting up an account and another moving part to break.
But there is a screen available in WHM: Account Information: View Bandwidth Usage.
It’s a lot of trouble to open up to WHM find that and click it. So I created direct browser bookmarks to our two cPanel servers which look like this:
http://91.162.85.74:2086/scripts/showbw
(change the numbers above for your own server’s WHM IP)
This gives you this very helpful chart to see who is in the yellow and red zone. I’ve already adjusted everyone’s plans to get them back into the white so sadly there’s no yellow or red on this chart.
disable suspending accounts bandwidth limit
If anyone knows how to get automated notices sent to server admin as well, please share the wealth.
New version of our plugin should cure the error message which was appearing on some hosts or sites with broken plugins:
"Toolbar set 'Foliovision' doesn’t exist"
We decided to not use that file anymore and moved all the configuration options for FCKEditor into inline JavaScript. That should fix the problem.
If you had this problem, download the latest version of our plugin (0.9.14) here. This new version fixes this problem and also some other bugs in file management.
At Foliovision, we consider one of our primary tasks keeping not just our inboxes free of spam but those of our high end clients as well. Especially as those clients use Blackberries and don't want to get spam on the road or all night long. At the same time, our high end clients are in businesses like real estate and insurance so false positives are a real danger. Fighting spam with no false positives is one of our toughest IT jobs.
Here's how we do it and how we did it, comparing SpamAssassin, Mailfoundry and SpamSieve (Apple only unfortunately, but there are Windows alternatives available, including the built-in filter in Thunderbird).
cPanel comes with SpamAssassin by default. SpamAssassin is powerful software but can come up with false positives. The default setup is 5. Moving it up to 8 or even 6 will reduce your false positives substantially. Viagra type spam comes in at 12 plus. At WiredTree, they have configured email with Exim to extensively use RBL (real time blackhole). This is so very important as a huge amount of spam doesn’t even reach SpamAssassin or your inbox. Unlike spam filtering, RBL filtering is quite accurate. Not only that but your legitimate correspondents will get a bounce notice so they can try and contact you via other means. Given the profluence of social networks (LinkedIn, Facebook, Twitter), contact forms and free email services (Gmail, Yahoo, Hotmail just to name the big three) if they can’t be bothered contacting you otherwise, you probably didn’t need the email.
The other huge advantage of RBL filtering is the amount of spam which you have to check by hand, goes way, way down.
Previously we used MailFoundry at CartikaHosting which was reasonably accurate but suffered from the following problems:
far too much spam gets through to the blocked list which means you have a lot of spam to comb by hand
far too many false positives which means you have to go through those lists, as MailFoundry learns from experience (i.e. if you allow false positives, the situation will get worse, unlike most installs of SpamAssassin where the chances of a false positive will stay the same)
a separate report for every mailbox, even when you redirect them to a single bulk mailbox. When you have a lot of emails to cover (my situation), getting 7 to 20 spam reports to check per day, leaves you with nearly 150 per week. I dreaded getting those MailFoundry reports. With SpamAssassin, all the spam gets filtered in the bulk mailbox and I have just a single mailbox full of spam to check occasionally and which can be sorted by from addresss, date or subject line.
After dumb SpamAssassin (non-learning), one can add a local learning spam filter on your primary computer (it won’t help with your secondary machines) to reduce spam still further. I use Michael Tsai’s SpamSieve and am very satisfied with it. Given the amount of email which goes through my computer, state of the art is worth the investment for me. Both Thunderbird and Apple Mail come with quite decent built-in Bayesian filters which would probably do for most people.
With your SpamAssassin settings up at 8, you won't have too many false positives (default 5 is dangerous) but with a local trainable filter (setting up and managing training filters in SpamAssassin is far too painful even on IMAP), you won't see much spam either and you can train your own ham (marginal spam) to pass your local filters.
The next nice trick that SpamAssassin has is the spam box. It will automatically file everything away where you don’t have to see it unless you want to. Spam will be removed from your inbox before it gets there. That’s very important for Blackberry and mobile mail users as the last thing you want on the road are notifications for spam. Moreover, on these mobile devices it’s not easy or inexpensive to run spam filtering. You can even route your spam to a secondary mailbox based on the subject line. In the secondary mailbox it will be directly routed to the spam folder based on the subject line, further reducing the number of spam boxes one has to check. The disadvantage is that any false positives will be much harder to find in one’s secondary mailbox (thousands of spam per week, in comparison to the tens of spam per week in my primary mailbox).
By leaving my bulk email in POP with SpamAssassin, smart filters and Bayesian filters, I end up with just a SpamAssassin spam box to check and my local SpamSieve filter. The best of both worlds.
This is a huge advantage over MailFoundry and its endless reports. I'm very happy to be away from Cartika Hosting's Hsphere/Mailfoundry combination and back on cPanel/SpamAssassin with RBL at Wiredtree.
You do want to be using SSL. Unencrypted connections are far too easily eavesdropped. On the other hand, it’s worth remembering that SSL only gets your login and email encrypted between your computer and your smtp server. Once your email hits the big pipes, it’s unencrypted again, vulnerable to whomever can get access to the transit points. A rogue operative in any ISP or fiber optic supplier could still siphon off huge amounts of data. Even if such a person existed, s/he would be unlikely to be able to regularly get all of your email though. However, random emails, especially if they traverse exotic territories with loose security could be grabbed.
Email is not private. Don’t forget that ever. Email is not private.
Thunderbird, Apple Mail, Outlook Express (and the hideous blue Windows Live Writer replacement) all offer SSL. But be sure not to check “Secure Authentication” if your particular host does not support it. Secure Authentication just has to do with security certificates and does not actually increase your security once you have an appropriately made self-signed certificate. Just be careful about authorising changes to your security certificate.
SSLIMAP will go across port 993. SSLPOP will go across port 995. Checking your port assignment is one way to be sure your email connection is encrypted. SMTP SSL will work across port 25 (and probably some others, but I've tested port 25).
SSL will get you most of the way there to reasonable privacy.
While you are at it, make sure that when you access webmail (with cPanel at http://yourdomain.com/webmail, the connection forwards to an https address.
Don’t forget that when you are visiting other login protected websites, unless the connection is https, you are handing over your login and password to the owner of the hotspot (if he cares to log it or take it). If you plan to spend a lot of time on hotspots as a traveller, you need more than SSL email. You need a VPN. Don’t go cheap or unbranded on your VPN supplier (setting up your own VPN is a big enough hassle that most small businesses should be outsourcing their VPN connections), as that organisation will have steady access to all of your communication and can keep really detailed logs. Unlike a single rogue hotspot, you will be using the VPN consistently over time and with a the same login.
These steps should not be considered security overkill, but just the basics.
The most important thing which you must know before starting the move is whether the mail account to be moved is POP or IMAP.
If the account is POP, your task is fairly straightforward.
You want to make sure that you move any unread move (mail from between the time your client last collected email and the time of the move is picked up and put on the new mail server). The best way to do that is to log into the old mail server and the new mail server via IMAP simultaneously. You will see what has been read and what hasn't. Just move the unread messages.
If you move the read messages, when your client logs in again via POP, he or she will have to sort through a 1000 or even 3000 archived messages in the inbox. Not fun.
We recommend using Apple Mail as the IMAP client as it's very easy and quick to set up. Windows Live Mail hides the IMAP accounts and folders and is ugly as sin. Thunderbird is very fiddly and exposes too many options but could do in a pinch or if you don't have a Mac handy.
After the move, there may also be a small lag while your client is only seeing the old server for reasons of DNS cache and won't get his or her new mail. For that reason, it's better to shut down the old mail server immediately after transferring the mail so he/she can't be picking up mail from both places at once.
For that reason I recommend doing the move at night at 3 in the morning, as your client has probably turned off his or her computer so the DNS will be renewed in the morning. If not, you'll have to ask them to flush their DNS cache (it's easy enough via GUI without IPconfig: turn on and off networking in the network control panel in Windows, Macs will flush the DNS automatically by switching network configurations). In the case, that even that is too technical, a restart will do the trick.
For a truly seamless move, it's essential that your host is using a modern convention for the mail login and smtp. All of our good hosts are using mail.domainname.com for POP, IMAP and SMTP. You will probably also be so lucky. If that's not the case, then you definitely have to involve the client in the move so that they will have the new login information.
At Foliovision, we proceed on the principle that our clients don't want to know about the IT unless they absolutely have to. They have work to take care of. It's up to us to sweat the details. And when we say sweat, we mean it. Ideally, they'd never notice that anything ever was changed or went wrong.
In principle, after moving from one host to another you should change all your passwords (the old ones are compromised from the previous host). In practice, I recommend keeping the passwords the same initially so that all of a client's automated logins will keep working. One doesn't want to be troubleshooting passwords and account moves simultaneously. As long as the passwords are the same, the move should be seamless.
Of course in most cases your clients will have to authorise the new server for email. Most will do so as a matter of course. (Get asked enough about security, you stop caring.)
If your client is already using IMAP, your task becomes much more sophisticated. If he or she is just using the standard IMAP mailboxes (Junk, Sent, Trash), your life is pretty simple. In this case, you simply move the contents of each mailbox (including Inbox) to the new account. When your client logs in, the messages will match and he/she will carry on work as before.
If your client has a sophisticated server side nest of mailboxes, you have a lot more work in front of you. First you have to duplicate the mailbox structure by hand and then drag the contents of each mailboxes over by hand. Dragging and dropping whole mailboxes won't work (at least in Apple Mail), as they are made into subfolders of the inbox.
In that case, you can look at zipping up the maildir folders, moving them over and resetting permissions. That leaves you no guarantee that the accounts are working properly. So if it's not a high volume mail move, I recommend a move by hand to be sure everything is working properly before you leave the job.
Here's some gotchas to look out for with a mail moves. Basically, POP ignores IMAP's read and unread flags.
For IMAP moves, careful not to touch unread mail on the server as the client may never see it (his/her email client will ignore read mail)
when POP picks up mail, they are marked as read in IMAP (bad).
if mail is marked as read in IMAP, it still gets downloaded via POP as unread (bad).
STEP BY STEP GUIDES
First check the mail server logs to see which clients log in via POP and which login via IMAP. It's all there in black and white with usernames. You need root access for this or submit a support ticket for shared hosting.
Here's the step by step guide for POP moves:
set up new mail account on new server
set up IMAP account for old account in Apple Mail
set up IMAP account for new account in Apple Mail
drag only the unread contents to the new account
archive the old account for a few days (by disabling the mailbox: don't forget to delete all these accounts within a week for privacy reasons)
Here's the step by step guide for IMAP moves:
set up new mail account on new server
set up IMAP account for old account in Apple Mail
set up IMAP account for new account in Apple Mail
recreate the mailbox structure of the existing IMAP account
drag all the contents to the new mailboxes one by one
archive the old account for a few days (by disabling the mailbox: don't forget to delete all these accounts within a week for privacy reasons)
As I've mentioned, I handle hundreds of messages per day for myself and my clients. I have separated my email from bulk email effectively now, but still found my computer sluggish.
The problem seemed to be around Apple Mail. I'm new to IMAP so I decided to dig deeper. I initially thought the issue was with Rules, as I have SpamSieve and many dozens of rules to deal with bulk email (things I might want to read but don't want in my InBox).
The problems turns out to be something else altogether:
Smart Mailboxes. Every time you get new mail your Smart Mailboxes folders update their unread counts: "Updating Smart Mailbox Unread Counts" is the message you will see in Mail's Activity Monitor.
Apple Mail
Smart Mailboxes
Spiking CPU
I have a lot of Smart Mailboxes (great feature, btw) which allow me to check how many leads each of my clients received this week and this year. At a glance, I can see how business is going (don't forget to weed out automated tests occasionally). Some of these mailboxes have many thousands of messages. Updating them takes a few seconds each.
These Smart Mailboxes get updated every time you get even one new email.
Normally I check my email manually only which means these updates don't happen very often and happen at a time when I'm working in Mail and am not surprised or bothered by 15 seconds of sluggishness. On the other hand, with my primary account as IMAP on IDLE that meant every time I received a single email all my spam filters and Smart Mailboxes updates ran each time. Given that even my primary email gets at least a couple of hundred emails per day, that's more workstops than I'm willing to put up with.
The simplest solution then would be to close Mail altogether when I'm not using it. This option doesn't appeal to me at all as I use Mail for reference and for writing messages even when I'm not checking for new messages.
First issue: with IDLE checked (if it works, it often doesn't), you will get new message pushed to you like it or not. Turn off IDLE
Second issue: make sure your general preferences are set to update only manually.
Third, pray.
This will probably stop IMAP from updating automatically, making life much better again. At least when Mail is a background application.
Preferences checking for new mail manually in Apple Mail
If you click into Mail and start looking at IMAP messages for every unread message you read, the whole cycle of Smart Mailbox updates start again. There is the feeling of sluggishness.
That's the price of using IMAP in Apple Mail apparently. There is no solution I can find, apart from deleting all your Smart Mailboxes. Which makes Apple Mail no better than any other Mail client, albeit a little bit prettier.
I thought I had a solution here, but in the end but just found a problem. Not even prayer will help here.
What we need is a way to turn Smart Mailboxes on and off (I don't need them all the time, I just need them when I'm in the mood to do a bit of analysis). Deleting them all and recreating them is not really an option. It took me weeks to refine them.
One way of dealing with this would be to disable Spotlight (which would stop smart filtering) but that would mean no advanced search function, something I use everyday.
The only workable solution is an on and off option (probably in right click) for Smart Filters. Or even by each Smart Filter with right click. I don't expect that from Apple. But perhaps someone can find a hidden preference that we can use from the command line. Otherwise one has to go back to POP (not as much of an issue with POP as there seems to be a delay before the filters update).
Are there no heavy Mail users at Apple, who use Smart Filters in their work and use IMAP too?
Wordpress 3.0 is out, finally. No doubt, many good things under the hood. I particularly like that multiuser is now available. For once, the default theme is not visually embarrassing, although no doubt that will change once we've seen Twenty Ten a hundred times on splogs.
One bad thing, people's expectations that Wordpress 3.0 is:
bug free
compatible with current plugins
No way guys. There's no way that most popular plugins are ready for 3.0. So unless you are willing to go on a severe plugin and functionality diet, just forget about Wordpress 3.0 for a couple of months.
It's the same deal when Apple releases a new version of their OS (back in classic days or with OS X). You don't want 10.6.0 or 10.6.1 or even 10.6.2. Applications don't work and are incompatible. The sweet spot to move is the .4 or the .5 iteration.
Frankly, I'm still running 10.5.8 on my main computer and have had less headaches as a consequence. I'm about ready to move, as all my software now has Snow Leopard updates and all of the other workarounds have been published by those who love to live on the bleeding edge. Still I might skip 10.6 altogether though and move to 10.7 when it's ready. My girlfriend's Macbook is running 10.6 and the only thing I'm missing out on are the built-in scanner drivers (works great with our HP C3180, unlike HP's own software). Oh and Acorn 2, which is the best photo edit tool I've seen outside of Photoshop.
The crazy thing are greedy clients who want the latest and greatest before the paint is even dry. We just finished a huge Typepad to Wordpress move using our current plugin set (there's a fair amount of moving parts to make a T2WP conversion seamless) and one gentleman is clamouring for his free upgrade to Wordpress 3.0.
First we are not part of Wordpress incorporated (that's called Automattic). Second, he is running a media site and he needs all those plugins which haven't been even tested yet on Wordpress 3.0.
One step at a time.
Fortunately this time, Matt has promised to take some time off from developing core code and Wordpress will distribute their efforts to improving the Wordpress.org site and resources. A much better idea.
Normally this is where I’d say we’re about to start work on 3.1, but we’re actually not. We’re going to take a release cycle off to focus on all of the things around WordPress....so much of our effort has been focused on the core software it hasn’t left much time for anything else. Over the next three months we’re going to split into ninja/pirate teams focused on different areas of the around-WordPress experience, including the showcase, Codex, forums, profiles, update and compatibility APIs, theme directory, plugin directory, mailing lists, core plugins, wordcamp.org
We have more than enough functionality. Just keep the API's running clean and keep improving the invisible code.
Thanks and congratulations to those who did make it and created the core of the latest Wordpress.
We'll be busy over the next couple of months getting all of our plugins up-to-date and compatible with 3.0. For our users, no worries, all of our current active plugins will be making the transition.
I thought it would be time to congratulate our former CTO John Godley but strangely John's not on the list yet either, despite having written the best set of plugins out there and now working for Automattic.
If like me, you are an old Eudora hand, you probably used POP reliably for decades before moving to Apple Mail and the possibility of troublefree IMAP use.
Apple Mail Setup IMAP: To Take the Leap Or Not?
You’ve probably also heard horror stories of unsynced and lost email from those who took the jump to IMAP in the 90’s. You prefer the security of local mail on POP for the following reasons:
your mail doesn’t spend much time on the cloud so there’s less possibility of it being read unless someone is actively tracking you
what’s on your computer is getting backed up by you so you have physical control of the data
Now however you may have a Mac Mini, a Macbook, Macbook Pro 17” portable desktop, a Windows 7 netbook, a Nokia N97 mini smartphone with keyboard, an iPhone and an iPad as well as a photo studio Hackintosh. Or five of the above at any given time.
Making sure you always have local up to date email with you is a hassle (you may be doing it by carrying a FireWire boot drive with you everywhere, but that doesn’t work on the netbook or the Hackintosh or the iPhone or the iPad or your Nokia N97mini either.
Working with the web interface is getting old as it means going through the same email twice, once online and once locally to get everything put away in the right folders.
If like me, you have a lot of custom sorting routines built in to Apple Mail, you certainly don’t want to give up those two hundred odd handbuilt rules.
But have no fear. All is not lost, even for email power users. You can smoothly move to IMAP for your important mail. I receive at least 600 emails per day, not counting SPAM. So if I can successfully make the POP to IMAP move, so anyone can. Fortunately, the vast majority just need to be filed for future reference.
The first thing to do is to separate essential email from non-essential email. If you’ve been using throwaway addresses and forwards (which you should have been doing all along), make sure all of those extra addresses end up at an address like Leave that non-essential mail in POP where it will be sorted automatically as it comes in and put in all your custom local folders.
With your primary address, you will have a lot less mail to deal with. You likely have just two goals for incoming mail: Archive or Delete. I recommend you keep your IMAP structure quite simple. That’s been enough for me.
What I’d like to see again, I archive after reading, what I don’t want to see again I delete and what I haven’t dealt with yet stays in the inbox for the moment for when I get back to my main machine.
Of course that means I can’t deal with all the less important email when on the road. But that’s no problem. When I’m on other assignments I’m just as happy not to see all those newsletters and can go after the bulk mail when I’m particularly bloody minded.
Now enjoy being able to really work with your email on every machine you own. I recommend Thunderbird (now has a widescreen viewing option) on Windows and Apple Mail on Mac. Come home with your work done.
Every once in a while, I’d recommend moving your archived email to a local folder. That way not too much email stays online and you also have the security of local backup.
local and imap folders archive
IMAP will change your life for the better. Enjoy mail on all your mobile devices. Maintain sorting for bulk mail.
Now that the essential prep for the move is done, I will post the nitty gritty technical details of POP to IMAP transitions.
Recently we've done a couple of Typepad to Wordpress conversions for a very nice Austrian PR and marketing expert by the name of Karin Schmollgruber. Karin's very up-to-date on social media and Facebook.
While we are quite active on LinkedIn and Twitter and various other social sites, we avoid Facebook like leprosy. Mark Zuckerberg has shown time and time again that he is not someone to be trusted with your data. In fact, the origins of Facebook themselves are dubious, he hijacked someone else's code and project.
Our fundamental objections to Facebook go even deeper and affect our relationship with Google as well (I won't use Gmail, although we do use Gcal and some Google docs at work and of course I use the webmaster tools as well). Basically, in the Soviet Union, the government spent a huge part of GDP on its security apparatus of KGB gumshoes and their paid and unpaid informants, maintaining huge filing cabinet in the Lubyanka on people of interest. In the US, the FBI did similar surveillance of Black Panthers, human rights and Indian groups, although these activities represented a much smaller part of US GDP.
Google and even more so Facebook is us doing totalitarian surveillance work on ourselves for free. Basically every time you add data to Facebook or use Gmail you are turning unpaid informant on yourself and your family. In the US the boundaries between the state security organs and your private data are so emaciated that there is no difference between the data lying in US database at Facebook or Gmail or on a government server. In December 2009 on CNBC, Eric Schmidt Google CEO came right out and said it:
If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place.
Imagine our amusement when Karin made Foliovision an ideal example of how to find fans on Facebook. Karin's video is in German but basically she got a package in the mail from us and was very happy. She'd like to friend us in Facebook but we don't have a fan page.
If you are a Foliovision client or friend after you pay for our work, we will send you a really nice polo shirt along with a thank you note. We spent a lot time of time choosing really good shirts and getting first class printing and we mean the thank you that we send. The thank you gift should reflect the same high quality standards we set in our work.
Austrian PR and social media Expert Karin Schmollgruber show off her Foliovision polo shirt
It's probably a great marketing gesture but what is not accounted for in terms of its effectiveness, is that it is sincere. We really appreciate our T2WP clients from Microsoft to ZenDesk and the Hollywood Reporter to solo weblogs like Karin, Richard Nikoley and Scott McLeod at CASTLE trusting us to take good care of their websites and to do really great work for them.
So if there was some way to get a company fan page without having a personal Facebook account we'll probably do it. In the meantime, thanks Karin for the shout out and we'd recommend marketing via Twitter and LinkedIn. We've also heard that Xing is very big in the German speaking world and more similar to LinkedIn than Facebook.
At this point, Facebook is an attempt to privatise the web inside of a closed ecosphere so we really want to keep our distance.
We'll keep supporting the EFF, doing great work for our clients and sending out polo shirts. And staying clear of Facebook.
If you work on a busy Wordpress sites in a shared hosting environment, you know how important is to keep the number of MySQL queries down as much as possible. Even if you are using some caching plugin, it's a matter of principle.
Today I was shocked to see that one of my Wordpress templates (it's based on Cutline template) is taking more than 100 queries on the index page. I was removing various parts of the template until I found that it's the the_tags() Wordpress template tag.
And here's a number of queries going on on my test site right now with this code:
Total Time: 78 database queries run in 0.014724969863892 seconds.
Finally I discovered that if I replace the_excerpt() with the_content() (that means the full articles are displayed, not just first few sentences), the queries go down to around 50. Is the_tags() not working with database cache and global PHP object until the whole content is shown?
I found the exact line in the_content() function which makes this happen and put it into my code, right before the_excerpt:
If you have tried to set up network backup on OS X and you ran into the message "the backup disk image could not be created", probably this article will help you.
OS X's TimeMachine software had native support for network backup until the OS X Leopard 10.5.2 was released. Apple had its own reasons for the decision to remove network backup, but many advanced users including us at Foliovision would still like to be able to back up over the network.
We have a bunch of Mac Minis in a mixed network of Linux and Windows computers. We'd like to use all our Minis for work and not for backup and use one of our older Linux towers to store the backup.
Fortunately Apple left us an option to turn-on network support for TimeMachine by running the following command in terminal:
We can finally see network hardrives in selection for backup places. The problem might seems to be solved. No dice: it is only the beginning.
If you select a network drive for backup, you will see a very familiar message:
the backup disk image could not be created
Google automatically completes the phrase as soon as you type "the backup d".
The real problem is in the file system which must be used for TimeMachine backup. TimeMachine only supports Mac OS Extended (Journaled) file system.
The solution is to create a place on the network, which will trick TimeMachine, into thinking that it holds data in Mac OS Extended (Journaled) file system. This trick is accomplished via copying a sparsebundle image (with special name) to a network share. You have to do following steps for to make TimeMachine successfully run a backup.
OVERVIEW
Enable network hardrive support in TimeMachine
Mount network shared place for backup data
Create a sparsebundle virtual image
Copy the sparsebundle virtual image to the network shared place
Set up TimeMachine for network backup
Optimisation and other information
1. Enable network hardrive support for the TimeMachine software
Network hardrive support for TimeMachine is turn off by default. For changing this fact we have to type following command to the terminal: defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
After that, you should see mounted network harddrive in TimeMachine's locations for backup. If it is not this case, probably a restart is needed.
2. Mount Network shared place for backup data
If you have a shared location for backup data on Linux server, you can map the directory as a drive to Mac in Finder application. You have to do following steps:
Click GO and then “connect to server” in finder menu.
Write following path if your sharing is based on protocol samba (SMB): SMB://<IP_ADDRESS_or_COMPUTER_NAME>/<NAME_of_SHARED_PLACE> (e.g. SMB://192.168.1.25/Backup)
3. Create a sparsebundle virtual image
This step is very important, because the TimeMachine doesn’t allow to backup data to a network drive, which has any file system except “Mac OS Extended (Journaled)”. So we have to create a disk image in "Mac OS Extended (Journaled)” format.
We have two ways to do it:
Open disk utility (Disk Utility) and create new virtual disk with following atributes:
Save As: <computer hostname>_<hex mac address of en0 interface>.sparsebundle (e.g. MacAlec_00ef9a048c4f.sparsebundle, if you forget to add .sparsebundle, it will be added automatically)
Volume Name: Backup of <computer hostname> (e.g. Backup of MacAlec)
Volume Size: the max amount of space you're going to set aside for backups. (The volume size is depends on amounts of backup data. I have chosen 150 GB)
Volume Format: Mac OS Extended (Journaled)
Encryption: None
Partitions: No partition map
Image Format: Sparse bundle disk image
Note: Disk utility automatically mounts the image to system volumes. We won’t need the image mounted, so we can safely unmount it.
The previous step can be done by one command in terminal: hdiutil create -size 150g -fs HFS+J -volname "Backup of MacAlec" MacAlec_00ef9a048c4f.sparsebundle
3. Copy the sparsebundle virtual image to the network shared place
You can copy the sparsebundle image to the network shared place using one of the following steps:
Copy the file to shared place by drag and drop operation in Finder application.
If you prefer to work with the terminal, you can type this instead: cp -r <sparsebundle_image_disc_location>/<computer hostname>_<hex mac address of en0 interface>.sparsebundle /Volumes/<NAME_of_SHARED_PLACE>/(e.g. cp -r /Users/Alec/Documents/MacAlec_00ef9a048c4f.sparsebundle /Volumes/Backup/)
Now we can safely delete the sparsebundle image copy on local computer once we are sure that it has been copied to the shared location.
Set up the TimeMachine for network backup
Open TimeMachine preferences (in SystemPreferences).
Click on Change Disk button for select the network drive for backup. (We have to select the "Backup"in our case.)
The first backup will start in two minutes. TimeMachine supports incremental backup, thus the first backup can take very long time (it is depends on network bandwidth and amount of backup data).
If you want to eject the mapped shared volume (/Volumes/Backup in our case), you have to do it within the two minutes countdown before backup starts or after backup is finished. TimeMachine has its own mechanism for mapping network hardrives, so a backup process isn't interrupted by your hard drive mounting or unmounting. TimeMachine will automatically mount the virtual sparsebundle image, when the backup starts. You will see Backup of <computer_name> as connected device on your desktop, so you won't need to manually mount the network hard drive later.
4. Optimisation and other information
in order to avoid long delays in backup process caused by Spotlight indexing, You should set the Spotlight application to not index the mapped network drive. Here's how to remove spotlight indexing for a hard drive:
Open up the Spotlight software preferences in system preferences window
Move to privacy options and add the network drive by plus symbol
If you'd like to back up a little less often, you can modify the file com.apple.backupd-auto.plist which is located in /System/Library/LaunhDaemons/ to change backup time interval. Open up the file in text editor and find the section:
<key>StartInterval</key>
<integer>3600</integer>
You should change the number 3600 to the number of seconds of your backup interval.
Result
I deliberately deleted some files to test backup reliability. This worked just fine. I was able to restore data from backup using TimeMachine's restore feature. TimeMachine restore is easy to use and lets you choose data from any date and hour. If you have followed the exact steps above, you should have a working network backup now on a non-Apple computer.
