As I’ve mentioned, I handle hundreds of messages per day for myself and my clients. I have separated my email from bulk email effectively now, but still found my computer sluggish.
The problem seemed to be around Apple Mail. I’m new to IMAP so I decided to dig deeper. I initially thought the issue was with Rules, as I have SpamSieve and many dozens of rules to deal with bulk email (things I might want to read but don’t want in my InBox).
The problems turns out to be something else altogether:
Smart Mailboxes. Every time you get new mail your Smart Mailboxes folders update their unread counts: “Updating Smart Mailbox Unread Counts” is the message you will see in Mail’s Activity Monitor.
Apple Mail
Smart Mailboxes
Spiking CPU
I have a lot of Smart Mailboxes (great feature, btw) which allow me to check how many leads each of my clients received this week and this year. At a glance, I can see how business is going (don’t forget to weed out automated tests occasionally). Some of these mailboxes have many thousands of messages. Updating them takes a few seconds each.
These Smart Mailboxes get updated every time you get even one new email.
Normally I check my email manually only which means these updates don’t happen very often and happen at a time when I’m working in Mail and am not surprised or bothered by 15 seconds of sluggishness. On the other hand, with my primary account as IMAP on IDLE that meant every time I received a single email all my spam filters and Smart Mailboxes updates ran each time. Given that even my primary email gets at least a couple of hundred emails per day, that’s more workstops than I’m willing to put up with.
The simplest solution then would be to close Mail altogether when I’m not using it. This option doesn’t appeal to me at all as I use Mail for reference and for writing messages even when I’m not checking for new messages.
- First issue: with IDLE checked (if it works, it often doesn’t), you will get new message pushed to you like it or not. Turn off IDLE
- Second issue: make sure your general preferences are set to update only manually.
- Third, pray.
This will probably stop IMAP from updating automatically, making life much better again. At least when Mail is a background application.
Preferences checking for new mail manually in Apple Mail
If you click into Mail and start looking at IMAP messages for every unread message you read, the whole cycle of Smart Mailbox updates start again. There is the feeling of sluggishness.
That’s the price of using IMAP in Apple Mail apparently. There is no solution I can find, apart from deleting all your Smart Mailboxes. Which makes Apple Mail no better than any other Mail client, albeit a little bit prettier.
I thought I had a solution here, but in the end but just found a problem. Not even prayer will help here.
What we need is a way to turn Smart Mailboxes on and off (I don’t need them all the time, I just need them when I’m in the mood to do a bit of analysis). Deleting them all and recreating them is not really an option. It took me weeks to refine them.
One way of dealing with this would be to disable Spotlight (which would stop smart filtering) but that would mean no advanced search function, something I use everyday.
The only workable solution is an on and off option (probably in right click) for Smart Filters. Or even by each Smart Filter with right click. I don’t expect that from Apple. But perhaps someone can find a hidden preference that we can use from the command line. Otherwise one has to go back to POP (not as much of an issue with POP as there seems to be a delay before the filters update).
Are there no heavy Mail users at Apple, who use Smart Filters in their work and use IMAP too?
Alec Kinnear
Alec has been helping businesses succeed online since 2000. Alec is an SEM expert with a background in advertising, as a former Head of Television for Grey Moscow and Senior Television Producer for Bates, Saatchi and Saatchi Russia.
I average about 7000 e-mails a day, use smart mailboxes, and don’t have this problem.
Set up:
Default email reader: Mail.app (4.3) Check for new messages: Every minute New messages sound: None Play sounds (unchecked) Doc unread count: Unread Add invitations to iCal: never Downloads folder: Downloads Remove unedited downloads: After message is deleted If outgoing server is unavail: Show list of alternate When searching include results from (only trash is checked)
Notes:
I often have two or three Mail.apps running against the same account. (one mobile, one at the office, and one at home).
iCal is set to never because you can’t ignore or delete an invitation. I get many invitations that are sent out to a list but that I never attend. This causes iCal to fill up with junk invitations.
Data on mail received for me during the last few days:
date received delivered
Jun 12 2010 6200 6234 Jun 13 2010 8246 8262 Jun 14 2010 9316 9376 Jun 15 2010 8301 8317 Jun 16 2010 8506 8514 Jun 17 2010 8543 8558 Jun 18 2010 8401 8415
Hi Jerry,
Thanks for sharing your detailed data. Do you get in Activity Monitor flashing data that all your Smart Mailboxes are being updated.
The reason I ask, is apparently the editors at Ars Technica had the same issues I do with Smart Mailboxes (and also ended up deleting them).
No, I never get that. In the interest of full disclosure I will add that because I’ve had no problems I haven’t looked for it either (until now).
I am running in 64bit mode, so perhaps that helps.
Systems:
8 core 3GHz Mac Pro 16GB with an Xserve RAID (home directories are located here as well as mail)
4 core 2.93 Mac Pro 8GB + 4 core 2.93 Mac Pro 6GB standby
I check from the iPhone as well but I don’t use push notification and of course, the iPhone doesn’t know about smart mailboxes.
Organization:
I use a few hundred folders and a lot of rules to categorize the mail and a few smart mailboxes to bring the latest items to my attention. Up until 10.6.3 only a 2GB mailbox was allowed so I spent a lot of time moving messages. That’s been fixed now with the switch from Cyrus to Dovecot.
The mail is archived in DEVONthink for easy searching.
My main complaint is the lack of a duplicate removal tool in Mail.app.
Hi Jerry,
You have a weird setup in that you are not filing messages (I have archives by year with some separate mailboxes, never even close to 2 GB per mailbox). All my mail is in Apple Mail, all 5 GB worth.
On top of that you are running on two Mac Pros. My primary office computers are a Core2Duo MacBook Pro and a Mac Mini with 6 and 4 GB of ram respectively. No slouches, but certainly not the power you have on deck, particularly with the RAID. I’m not surprised you are not noticing the scanning.
Are you sure you are using IMAP mode? I think if you are on POP with your setup you won’t notice anything (I managed to use POP for years without figuring out what those high CPU cycles from Mail were).
Any solution will come from someone using IMAP really seriously. I still have my busiest mailboxes (although less important) on POP.
LOL
I get at least 3GB of mail a month. I don’t think Mail.app can handle the amount of mail I have stored, and it won’t search content remotely either (only TO, FROM, and SUBJECT so I have to have some other way to search–saving a copy of the messages on the remote system, which is how Apple wants to do it, isn’t really an option).
And sometimes they are stretched to the limit (because I do more than just mail).
Absolutely. I have POP is turned off because I don’t want messages downloaded (and yes, I know that you can leave a copy on the server). I only allow access via IMAP.
Well Jerry, I thought I had email volume problems with 500 messages a day. I’m glad I don’t have to receive your email.
Seriously thought Mac Pros are not a good baseline for performance. On the other hand, you are probably running Snow Leopard and I’m running Leopard (Mail.app 3.6). Perhaps IMAP and Smart Mailboxes are fixed in Snow Leopard. I doubt it though.
Yes, I’m running 10.6.4. I haven’t changed the smart mailboxes since 10.5.8 though.
Snow Leopard made a big difference in the performance of Mail.app in general but I attribute that to running in 64bit rather than 32bit mode because the improvement started with 10.6.3 (the first SL release I upgraded to on the main system). I can actually move 1500 messages from one folder to another, move another 1500, and Mail.app is still responsive. Before 10.6 I had to wait after each move.
I haven’t been able to tell any specific difference in the Smart Mailbox performance but I wasn’t looking for any difference either.
And I guess using a Mac Pro as a baseline depends on what you do and what you’re used to. Before migrating to Apple I used SGI systems for over ten years, 1995-2007 (until they got into trouble and stopped supporting desktop systems and applications). Mac Pros are a bit more expensive and don’t have the same performance, application stability, or reliability (uptime), and I really miss the Shelf, but they have a (much) better range of software and almost as low a requirement for system administration.
Thanks for the detailed notes on Snow Leopard. To be honest, much as I like Apple systems, I’m waiting for Linux to smarten up and be more like SGI.
Thanks. I ran across this post while googling for another problem related to mail.app. I had assumed smart mailboxes were just CPU time wasters in general and had never made the connection to IMAP IDLE.
Alec, I found your site on this nasty problem, because on my iMac it’s driving me crazy, too. Thank you for writing about this bug.
My setup contains about 400.000 archived mails, and every time I change any status of any mail it will go into this “Updating Smart Mailbox Unread Counts” cycle. I do have only 7 smart mailboxes, have turned off any IDLE options, but still it takes from 20 seconds to a minute with 100% CPU. Well, I have 4 CPUs on my iMac, but it’s a performance hog, nevertheless.
Have you already switched to Lion and maybe there is relief?
Hi Robert,
Thanks for your note.
I haven’t switched to Lion and may never do so (I don’t like the direction Apple is taking in security, privacy or OS interface).
The only viable thing I was able to do (despite the usefulness of smart mailboxes) was to turn them off. Anything else cripples the computer.
I’ve not been able to replicate the functionality of the smart mailboxes and there are things I’d like to monitor daily for our clients which I’m not able to do.
Good luck!