WordPress Speed Test 2012: WP Super Cache vs HyperCache

WordPress cache plugins were in really sad shape by the summer of 2012. WordPress 3.3 and WordPress 3.4 have changed some important parts of these plugins and they started to collapse. The first one to go was W3 Total Cache (at WordPress.org) which has been getting about 50% broken ratings since last fall. After spending months learning how to use W3 Total Cache just right (it's a complicated beast), we had to pull it off all our sites. Fortunately the much simpler WP Super Cache (at WordPress.org) remained bullet proof.

This spring and summer WP Super Cache broke too. Garbage collection does not work reliably any more, leading to people getting served week old pages. Donncha first suggested it's impossible to make WP Super Cache work for all hosting out of the box. Strange as for more than five years WP Super Cache did just that. The real answer came a bit later. WP Super Cache is a free plugin and Donncha just doesn't have time anymore.

I have hardly any time to devote to this any more. One of the dangers of becoming a father I'm afraid.

We've struggled to get garbage collection working properly to keep our clients' sites cached but reliably up to date. For the moment garbage collection still won't run reliably on at least Informed Comment. As JuanCole.com is one of the most read political sites in the world, this is something which needed to be fixed right away. It turns out there is a third WordPress cache plugin out there, Hyper Cache (at WordPress.org). We know the author of this plugin Stefano Lissa from his Newsletter Pro plugin which we use for a few clients. Lissa's code is good, he can handle sophisticated tasks while keeping the code structured enough for external teams to find and repair bugs and he answers his email.

So while HyperCache is less well-known than the big two, it seemed worth a try. We think long and hard before changing horses on core functionality like caching as we have years of experience with our core plugins. Usually our philosophy is better the devil you know is better than the one you don't.

In addition, WP Super Cache has mod_rewrite capability which means no PHP has to run (and hence presumably no CPU) to serve cached pages. We put a high value on Super Cache's capability to bypass PHP processing. But to our surprise testing last year had shown to Hyper Cache to be a bit faster than WP Super Cache. Other testing had shown Hyper Cache competitive. What is also worth noting from the other two tests is that in a comparable environment W3 Total Cache is in no way superior to either WP Super Cache and Hyper Cache, just more complicated. Complicated is bad: KISS is the best development rule ever written.

We still didn't really believe these results and thought there must be something strange in the test environment. We wanted to test against a real site and a real server that had been online.

We still had a testbed server available with Informed Comment on it and no traffic. The nice thing about this test bed is that it is a very limited 768 MB VPS with bare bones Apache and mod_php on it. I.e. we knew that if we gave it a good effort we'd be able to saturate it with proper external testing from LoadImpact. Our main dedicated servers with nginx would cost hundreds of dollars per test instead of $15/test. Based on past testing, we far prefer real web traffic than synthetic benchmarks like ApacheBench.

Here's what we found with 500 concurrent connection test. First the results for WP Super Cache.

WP Super Cache 500 concurrent users load time graph LoadImpact
WP Super Cache 500 concurrent users load time graph LoadImpact

Now from the challenger HyperCache.

HyperCache 500 concurrent users load time graph LoadImpact
HyperCache 500 concurrent users load time graph LoadImpact

Basically the results are identical. Both plugins allow the post to be downloaded at a fairly constant 1.8 seconds per load.

But HyperCache did not require any mod_rewrite rules.

Martin and I were convinced that HyperCache probably used a lot more CPU so we should stick with WP SuperCache. Here are CPU usage and traffic graphs for Hyper Cache:

HyperCache 500 concurrent users CPU usage
HyperCache 500 concurrent users CPU usage
HyperCache 500 concurrent users Traffic
HyperCache 500 concurrent users Traffic

Not looking so good. A pretty big spike in CPU there. We were none too impressed. But here's what we saw from WP Super Cache:

WP Super Cache 500 concurrent users CPU usage
WP Super Cache 500 concurrent users CPU usage
WP Super Cache 500 concurrent users traffic
WP Super Cache 500 concurrent users traffic

Astonishing. While the traffic graph looks more or less the same, it takes WP Super Cache much, much longer to recover from an overload. The CPU spikes above 10 and doesn't come back down for another five minutes. mod_rewrite at least in the WP Super Cache implementation is not more efficient. It takes a long time to recover.

If your server ever does overload, you want HyperCache on your side. As soon as the load goes down, a site with HyperCache will start serving quickly again until overloaded.

Three hundred concurrent users (we were using LoadImpact's lighter version VBU and SBU: virtual browser units as opposed to simulated browser units) is actually quite a few visitors. In real world terms is about 100 people loading a page simultaneously which on a site where visitors have something to read is more like 2000 to 5000 visitors on your site at the same time with no slow down. A normal busy site would have about 20,000 visitors/day (super sites like HuffingtonPost.com or Guardian.co.uk are an entirely different magnitude and require very different technology than simple caching).

What we also like about HyperCache is its simplicity. We've been having issues with mobile caching with both WP Super Cache and W3 Total Cache. It will be very easy for us to solve these issues with HyperCache. Instead of a tabbed interface like W3 Total Cache or WP Super Cache (easy and advanced), all the options are on a single page, vertically laid out. The options are neither too simple nor too complicated and mainly self-explanatory. There is a minimum amount of decision making required. Most options are for simple quantities like which mobile user agents to divert to the mobile version.

On the other hand, proper caching is a tricky matter (otherwise why have we had to revise our procedures and change our caching plugin of choice three times in the last two years: WP Super Cache --> W3 Total Cache --> --> WP Super Cache --> HyperCache). If you aren't a programmer or a web developer, you'd do well to have someone experienced look at your hosting and install your caching for you. Your chances of successful plug and play do go up considerably with HyperCache. You definitely need to use a caching plugin. We found that servers can handle at least 10x as much traffic (more like 20x or 50x when properly set up) with caching as without.

If you do want to keep using WP Super Cache, we suggest you follow Donncha's advice and downgrade to version .9.9.9.9 or earlier. Available at the WordPress repository. Then just don't update it.

We hope this WordPress caching plugins test helps you too. As we integrate HyperCache into our dedicated server and nginx environment we will publish updates of anything unusual we find and/or helpful tips.

This entry was posted in SEO, WordPress and tagged , , , , , . Bookmark the permalink.

14 Responses to WordPress Speed Test 2012: WP Super Cache vs HyperCache

  1. John K

    Thanks for the info. Super Cache seems to have issues with WP 3.5 as well. Expired pages weren’t being deleted and pages weren’t being cached, so I switched to Hyper Cache and all seems well, knock on wood. I do like the simplicity as well, not that Super Cache was that complicated.

  2. Stefano Lissa

    Hi, thank you for your really complet analysis! Great!

    When you cite the usage of the .htaccess and the rewrite rules… time ago I made a test discovering that if the rules are too much (cookies checking with regular expressions and so on) the php was more performant!

    Initially it seemed strange to me, but probably the rewrite engine is not so optimized as a piece of php script probably compiled and kept in memory by accelerators.

  3. Hello Stefano,

    in our test of WP Super Cache, we used a normal Apache MPM Worker with PHP FastCGI, no accelerator on the PHP.

    Perhaps the results will be more in favor of WP Super Cache if you use Nginx, as it uses config files instead of slow .htaccess which has to be parsed on each request.

    But you can always get APC and that might help your plugin even more.

    Thanks,
    Martin

  4. Rolf

    Sounds great, so I will now change from W3TC to HyperCache too. But then another question comes: Which plugin for css and js combine/minify is recomended? Do you have any idea or experience to use best with HyperCache?

  5. Hi Rolf,

    Thanks for stopping by and for your questions.

    Yes, give HyperCache a try, it’s really super. We’ve been contributing some code to make it even better.

    For minifying, we’ve been using Minify (see our writeup of WordPress performance with Minify vs Cloudflare). BetterMinify looks pretty good as well but we’ve been happy with the original.

    Let us know how your own real world testing goes if you get the chance.

    Making the web work for you, Alec

    PS. You might like to try adding a Gravatar which would then show up here too (and lots of other places). Faces make the web a better and more personal place.

  6. Rolf

    Hi Alex,

    thanks. Hyper Cache works fine on 2 of my blogs, but the minifys don’t. So after testing several other minifys with no success I went back to W3 Total Cache, but deactivated the page and browser cache. Minify in W3TC works very good and 100% automatic, so perhaps it is a little overhead, but W3TC for minify and db cache and Hyper Cache as page cache works fine.

  7. Hayden James

    Please remember that use of .htaccess should be avoided… a LAST resort. Unless on a shared hosting plan or lack server experience. Also see Apache’s documentation about when NOT to use .htaccess:
    http://httpd.apache.org/docs/2.2/howto/htaccess.html#when

    This is why I was surprised by “HyperCache did not require any mod_rewrite rules. Martin and I were convinced that HyperCache probably used a lot more CPU so we should stick with WP SuperCache.”

    Hyper Cache NOT using mod_rewrite via .htaccess is a GREAT thing. :) That why its not a CPU hog, because Apache must load and continue to reload .htaccess on EVERY request. On servers with millions of page views this is unnecessary CPU overhead.

    If not on Nginx, it would be also wise to strip down Apache:
    http://haydenjames.io/strip-apache-improve-performance-memory-efficiency/

    Thanks for the article!

  8. Hello Hayden,

    thank you for the interesting links, however how can you run WordPress on Apache without .htaccess? That’s a thing to consider too.

    Indeed – we have a lot better experience with NginX. You have to write all your rewrites into the config files, but that’s only problem with WP Super Cache in rewrite mode, as that uses a lot of these rules. Hyper Cache runs with just the basic WordPress rewrite rules and doesn’t require anything else.

    I wonder if WP Super Cache in the PHP mode is just as efficient, but we had all kinds of issues back when we were using it, like Garbage collector not working out of the box. So I don’t see any reason to switch back to using it.

    It would be nice to see some actual comparison of your Stripped down Apache with normal Apache and NginX, but we also like NginX as we don’t have to fiddle with it too much.

    Thanks,
    Martin

  9. Hayden James

    Hi Martin,

    You run wordpress without .htaccess by using the following two steps:
    (note this is how Apache devs intend their Http server to be used)

    Step 1) Cut an pasting the WordPress permalink .htaccess rules into Apache conf file.

    Step 2) This is documented clearly in Apache docs (link posted above):
    “When AllowOverride is set to allow the use of .htaccess files, httpd will look in every directory for .htaccess files. Thus, permitting .htaccess files causes a performance hit, whether or not you actually even use them! Also, the .htaccess file is loaded every time a document is requested.”

    Thus, after step 1 you will then set AllowOverride to none. Also, best security practice.

    Sure thing, I will update my blog with more related articles soon. I just have not had the time.

    Thanks for following up.

  10. Pingback: WordPress Performance: Stop using .htaccess! Here's how & why to disable it. | Hayden James

  11. Hello Hayden,

    that’s interesting and actually a bit eye-opening – not to use the extensions which are not needed. I wonder why web hosts don’t use that reduced config on their servers, specially if you order VPS and it just doesn’t run properly out of the box and you have to tweak the Apache all the time.

    However I also have to say that we like the NginX config files more than the Apache virtual host/conf files.

    Thanks,
    Martin

  12. Hayden James

    Hi Marin,

    My thought too. I guess web hosts (not all, but most) LOVE the bloat. because it results in more frequent hosting plan upgrades/more money. Then there are other hosts motivated by ensuring better compatibility, if they strip Apache down to bare bones then they will have many support requests to add modules. :/ So this is best done after website(s) are setup on a non-shared server, so that only unused modules are removed.

    Yes love Nginx also. It also great that for those who don’t want to leave Apache because of compatibility or simplicity, they can easily run Nginx as proxy in front of Apache and it’s basically invisible. And as you’ve said right out the box Nginx is super fast! (can be master faster too but tweaking is only required for extreme high traffic needs)

    That said, I’m a BIG BIG fan of Apache. Apache 2.4 “when setup right” can run just as fast as Nginx. No, I have no benchmarks published. :) but we shouldn’t put faith in anyone’s benchmarks but our own. Except companies like phoronix.com who leaves so much detail about benchmark setups, that you can’t pick apart or question results very easily.

    That said, I look forward to seeing similar benchmark posts like this from you. It can be times taking, but so useful esp when all benchmark setup details are disclosed.

  13. Hi Hayden,

    Let’s try and do a test of tweaked Apache for WordPress against Nginx. We have the tweaked Nginx beds at every size level ready.

    Can you set up and configure a stripped down Apache 2.4 VPS or dedicated box on which we could hammer WordPress with some serious load testing?

    Apache 2.4 has to continue to run with standard WordPress or it is not a fair test (if you have to rewrite the CMS to get it to run fast then you aren’t really testing the server).

    I’ll throw a spanner into the works and mention that the only server I really like is Litespeed which runs fast and reliably with minimal tweaking. It’s also $800/box but that amortises quickly. I wouldn’t run a cPanel server without Litespeed (it would be another interesting test to see your minimal configuration Apache 2.4 for cPanel).

    Cordial regards, Alec

  14. Jaydul

    I want use hyper cache with CDN
    I need use hyper cache also need CDN

    Please help

Leave a Reply