WordPress cache plugins were in really sad shape by the summer of 2012. WordPress 3.3 and WordPress 3.4 have changed some important parts of these plugins and they started to collapse. The first one to go was W3 Total Cache (at WordPress.org) which has been getting about 50% broken ratings since last fall. After spending months learning how to use W3 Total Cache just right (it's a complicated beast), we had to pull it off all our sites. Fortunately the much simpler WP Super Cache (at WordPress.org) remained bullet proof.
This spring and summer WP Super Cache broke too. Garbage collection does not work reliably any more, leading to people getting served week old pages. Donncha first suggested it's impossible to make WP Super Cache work for all hosting out of the box. Strange as for more than five years WP Super Cache did just that. The real answer came a bit later. WP Super Cache is a free plugin and Donncha just doesn't have time anymore.
I have hardly any time to devote to this any more. One of the dangers of becoming a father I'm afraid.
We've struggled to get garbage collection working properly to keep our clients' sites cached but reliably up to date. For the moment garbage collection still won't run reliably on at least Informed Comment. As JuanCole.com is one of the most read political sites in the world, this is something which needed to be fixed right away. It turns out there is a third WordPress cache plugin out there, Hyper Cache (at WordPress.org). We know the author of this plugin Stefano Lissa from his Newsletter Pro plugin which we use for a few clients. Lissa's code is good, he can handle sophisticated tasks while keeping the code structured enough for external teams to find and repair bugs and he answers his email.
So while HyperCache is less well-known than the big two, it seemed worth a try. We think long and hard before changing horses on core functionality like caching as we have years of experience with our core plugins. Usually our philosophy is better the devil you know is better than the one you don't.
In addition, WP Super Cache has mod_rewrite capability which means no PHP has to run (and hence presumably no CPU) to serve cached pages. We put a high value on Super Cache's capability to bypass PHP processing. But to our surprise testing last year had shown to Hyper Cache to be a bit faster than WP Super Cache. Other testing had shown Hyper Cache competitive. What is also worth noting from the other two tests is that in a comparable environment W3 Total Cache is in no way superior to either WP Super Cache and Hyper Cache, just more complicated. Complicated is bad: KISS is the best development rule ever written.
We still didn't really believe these results and thought there must be something strange in the test environment. We wanted to test against a real site and a real server that had been online.
We still had a testbed server available with Informed Comment on it and no traffic. The nice thing about this test bed is that it is a very limited 768 MB VPS with bare bones Apache and mod_php on it. I.e. we knew that if we gave it a good effort we'd be able to saturate it with proper external testing from LoadImpact. Our main dedicated servers with nginx would cost hundreds of dollars per test instead of $15/test. Based on past testing, we far prefer real web traffic than synthetic benchmarks like ApacheBench.
Here's what we found with 500 concurrent connection test. First the results for WP Super Cache.
Now from the challenger HyperCache.
Basically the results are identical. Both plugins allow the post to be downloaded at a fairly constant 1.8 seconds per load.
But HyperCache did not require any mod_rewrite rules.
Martin and I were convinced that HyperCache probably used a lot more CPU so we should stick with WP SuperCache. Here are CPU usage and traffic graphs for Hyper Cache:
Not looking so good. A pretty big spike in CPU there. We were none too impressed. But here's what we saw from WP Super Cache:
Astonishing. While the traffic graph looks more or less the same, it takes WP Super Cache much, much longer to recover from an overload. The CPU spikes above 10 and doesn't come back down for another five minutes. mod_rewrite at least in the WP Super Cache implementation is not more efficient. It takes a long time to recover.
If your server ever does overload, you want HyperCache on your side. As soon as the load goes down, a site with HyperCache will start serving quickly again until overloaded.
Three hundred concurrent users (we were using LoadImpact's lighter version VBU and SBU: virtual browser units as opposed to simulated browser units) is actually quite a few visitors. In real world terms is about 100 people loading a page simultaneously which on a site where visitors have something to read is more like 2000 to 5000 visitors on your site at the same time with no slow down. A normal busy site would have about 20,000 visitors/day (super sites like HuffingtonPost.com or Guardian.co.uk are an entirely different magnitude and require very different technology than simple caching).
What we also like about HyperCache is its simplicity. We've been having issues with mobile caching with both WP Super Cache and W3 Total Cache. It will be very easy for us to solve these issues with HyperCache. Instead of a tabbed interface like W3 Total Cache or WP Super Cache (easy and advanced), all the options are on a single page, vertically laid out. The options are neither too simple nor too complicated and mainly self-explanatory. There is a minimum amount of decision making required. Most options are for simple quantities like which mobile user agents to divert to the mobile version.
On the other hand, proper caching is a tricky matter (otherwise why have we had to revise our procedures and change our caching plugin of choice three times in the last two years: WP Super Cache --> W3 Total Cache --> --> WP Super Cache --> HyperCache). If you aren't a programmer or a web developer, you'd do well to have someone experienced look at your hosting and install your caching for you. Your chances of successful plug and play do go up considerably with HyperCache. You definitely need to use a caching plugin. We found that servers can handle at least 10x as much traffic (more like 20x or 50x when properly set up) with caching as without.
If you do want to keep using WP Super Cache, we suggest you follow Donncha's advice and downgrade to version .22.214.171.124 or earlier. Available at the WordPress repository. Then just don't update it.
We hope this WordPress caching plugins test helps you too. As we integrate HyperCache into our dedicated server and nginx environment we will publish updates of anything unusual we find and/or helpful tips.