• Skip to content
  • Skip to primary sidebar
  • Skip to footer

Foliovision

Making the web work for you

Main navigation

  • Weblog
    • FV Player
    • WordPress
    • Video of the Week
    • Case Studies
    • Business
  • About
    • Testimonials
    • Meet the Team
    • We Support
    • Careers
    • Contact
    • Pricing
  • Products
  • Support
    • FV Player Docs
    • Pro Support
  • Login
  • Basket is empty
Affordable VAST/VPAID for Wordpress has arrived. Serve ads with your videos starting today!

Why WP Rocket fails on high traffic sites and how to fix it

6 December 2017 / Alec Kinnear / 2 Comments

We are big users of WP Rocket (licensed through 2020). WP Rocket works great on our clients' normal busy sites (up to 5 million visitors/month). We run one very high traffic network with up to 40 million visitors in one week or even a million visitors/hour.

On this site, WP Rocket fails miserably. When new articles are posted WP Rocket purges the cache of too many different pages:

  • all the category archives to which the page belongs
  • related date archives
  • even the previous post and previous post in the same category (this is to keep the prev/next links on the posts working, if the post is updated it clears the next post too)
  • feeds
  • and of course the homepage

And all of the above happens if a post is added, updated or a comment is posted. So when you have people writing new comments every second you end up with a lot of cache purges on the pages where the information about new comment is not so important - new comments should only purge that post cache and nothing else.

Also - the full page cache is purged anytime a new user registers!

So we ended up implementing a queue for the cache purge actions. That way we are able to limit the amount of purges of any given URL to once per 10 seconds or even 1 minute for the less important pages.

Another problem is that once the cache entry for any given URL is purged, it has to be visited again to make sure it's in cache. On our heavily optimized site with PHP 7 + MariaDB + Redis and all sorts of fragment caching this takes 200 - 300 ms.

So if you have hundreds of visitors coming in every seconds, these requests come in and backup the pipeline, clogging our powerful server like a drain. All of them start to generate the cache file at once resulting in a disaster.

The proper fix for WP Rocket would be to not purge the URL by deleting it and then hoping the first visitor will re-generate the cache entry, but generating a new version of it on background and storing that in cache. That way at no point would visitors then be facing a missing cache and the server would not overload. 

Of course we couldn't leave a site to crash like this so we had to configure NginX Microcaching (fastcgi_cache). It took us some time to perfect our caching rules for it, but it's really fast and prevents race conditions altogether.

It's a pity though that WP Rocket isn't tuned for really high traffic as the benefits would be huge. Specially if it's free competitor WP Super Cache has the solution - check it's "Cache rebuild" option - it makes sure a supercache file is served to anonymous users while a new file is being generated. Perhaps we should give that one a try.

As we have a pro video plugin (WP.org) and a baker's dozen free plugins to take care of and we are generally happy with WP Rocket, we have no plans to take caching into our own hands. I sincerely hope that Jean-Baptiste and Caspar will add this improvement, at least as an option, for high traffic sites. We did report the issue to WP Rocket GitHub repository but got no answer for 5 months now.

Alec Kinnear

Alec Kinnear

Alec has been helping businesses succeed online since 2000. Alec is an SEM expert with a background in advertising, as a former Head of Television for Grey Moscow and Senior Television Producer for Bates, Saatchi and Saatchi Russia.

Share6
Tweet
Share
6 Shares

Categories: WordPress Tags: caching, wp rocket, wp super cache

Related Posts

  1. WordPress Caching Drag Race: Hyper Cache vs. WP Rocket

  2. Why WPEngine caching doesn’t work with eCommerce sites

  3. WordPress Speed Test 2012: WP Super Cache vs HyperCache

    WordPress Speed Test 2012: WP Super Cache vs HyperCache

Reader Interactions

Comments

  1. Amir Emami 29 March 2019 at 6:44 pm

    Hi

    Thanks for this article, but the feature you are talking about isn’t “Preload” option wp rocket?

    According to wp rocket:

    Preload: When you enable preloading WP Rocket will generate the cache starting with the links on your homepage followed by the sitemaps you specify. Preloading is automatically triggered when you add or update content and can also be manually triggered from the admin bar or from the WP Rocket Dashboard.

    Reply
  2. Martin Vicenik 9 April 2019 at 11:32 am

    Hello Amir,

    the preload is nice, but it doesn’t help with the issue of a large spike of uncached hits when some cache entry is purged. There is no way of sending the expired file to the visitor as the new cached entry is being generated.

    Thanks,
    Martin

    Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

You can click here to Subscribe without commenting

Primary Sidebar

Categories

  • Business
  • Cameras
  • Case Studies
  • Design
  • Flowplayer
  • Internet Marketing
  • IT
  • Life
  • SEO
  • Slovak
  • Video of the Week
  • WordPress

Footer

Our Plugins

  • FV WordPress Flowplayer
  • FV Thoughtful Comments
  • FV Simpler SEO
  • FV Antispam
  • FV Gravatar Cache
  • FV Testimonials

Free Tools

  • Pandoc Online
  • Article spinner
  • WordPress Password Finder
  • Delete LinkedIn Account
  • Responsive Design Calculator
Foliovision logo
All materials © 2021 Foliovision s.r.o. | Panská 12 - 81101 Bratislava - Slovakia | info@foliovision.com
  • This Site Uses Cookies
  • Privacy Policy
  • Terms of Service
  • Site Map
  • Contact
  • Tel. +1 518 412 4600