… and what can be done about it
Many successful WordPress site owners have moved their sites over to WPEngine for their high performance and high speed even under very heavy traffic.
WP Engine is able to provide this kind of speed thanks to their ”hand-built a WordPress-specific EverCache system” and “a fully-managed CDN service” (for more info see WP Engine’s articles on speed and infrastructure).
However, digging deeper reveals, that Cookies and PHP Sessions don’t work as one would expect or want:
- PHP Sessions are actually incompatible with WP Engine page caching. They list a number of security reasons for that, including Session Poisoning, Session Fixation, and Session ID Hijacking.
Sessions generally aren’t compatible with page caching, because the arguments at the end of the URL that carry
$_SESSIONinformation are dropped. This means that if you (or your users) are logged into WordPress, Sessions will likely work as expected. Otherwise, Session data will appear to be empty. - Cookies can be used but need to be addressed exclusively via JavaScript or WordPress specific Ajax calls.
Cookies can still be used with page caching. However, they need to be handled predominantly with Javascript instead of PHP. If you try to use PHP to read cookies, it will likely only display an empty cookie array.
This does not mean that you cannot use PHP at all with cookies. If you want to use PHP, then you will need to use Ajax to make requests to the server and run PHP code. WordPress has native support for Ajax functionality, and also makes that functionality extensible. For more information on using Ajax with WordPress, see the Codex entries for Ajax and Ajax in Plugins.
All this sounds fine until you figure out that:
Plugin developers don’t code with systems in mind that use such restrictive page caching. Which means that your average eCommerce system (like WooCommerce) will not be able to function without cookies.
WPEngine’s standpoint is that
- either you should use code that handles cookies with JavaScript or WordPress Ajax
- or you need to ask WPEngine Support to turn cache off on your eCommerce system’s front-end pages
Because these settings can vary from site to site, we currently require you to submit a ticket to request pages on your site be un-cached.
For our customers who are set up on clusters, we would have to completely change how our load balancers work, just to make sure that $_SESSION variables were available between different servers
Now let’s imagine you have a burst of visits to your eCommerce site and you went with the second option – turning cache off.
Well, nicely put, your business will have a crisis as even WPEngine’s environment cannot handle a huge number of visits, if caching is off and they hadn’t prepared for the load in advance.
502 error on a WP engine site
The above 502 error shows you just such a crisis. This happened to our client Dave McGeady this weekend. Dave’s dream came true: his small business was featured in a full page article in the Daily Mirror national newspaper in the UK. Wyldsson Elite Nutrition has had a sudden spike in visitors. As Dave explains.
Rory McIlroy (golfer) was eating my products on television while he was playing golf on Sunday.
This is truly a great media mention. Such a huge amount of traffic to your site sounds like a great boost for your business, right? Well, not if your site cannot handle it.
Rory McIlroy eating wyldsson nuts on television
Rory McIlroy eating wyldsson nuts on television
A small business can only hope for a few such events per year. To lose that traffic to 502 errors is worse than frustrating.
The only solutions in such a case are, as far as I can tell, getting moved to a stronger server or node or disabling plugins you don’t need desperately and WordPress functionality like (sending of emails), while the huge load lasts. Preferably both.
If you are a developer or have a team of developers and want to go with the first option – handle cookies in PHP with Ajax, here is a neat article about how to set it all up:
WP Engine Failed Dave McGeady: What WP Engine Should Have Done Better
The above is a cool developer’s perspective written by Karol. I’m Alec the founder and creative director at Foliovision. As a business owner, I think a bit differently: I’d be furious if my (relatively) expensive hosting went down under load. My expectation would be completely different. Instead of making excuses for the failure as did John at WP Engine, shifting blame to payment provider Paymill, WP Engine should have moved Dave’s site immediately to a high availability node and worked with Dave through the spike to keep his site up.
If you don’t know how WP Engine bills, here’s how it works: you pay per visitor. Therefore by keeping Dave’s site up, WP Engine would make more money. WP Engine’s services are anything but free: each visitor costs .00065 cents which adds up quickly: two million visitors would cost $1300/month. This is not huge traffic: two million visitors/month is just 60K odd visitors per day.
Hopefully WP Engine doesn’t have the temerity to try to bill Dave for the visitors to whom they served 502 pages, throwing out high value traffic and a one time opportunity forever. I’m worried they might as billing is based on IP addresses and these failing IP addresses are still IP addressed.
Sure WP Engine could insist Dave take some steps to lower the load but throwing his site down the memory hole during his short term spike is bad business and very bad manners.
Long Term Solution to WP Engine eCommerce Woes
WP Engine expecting all site owners to run their ecommerce according to their narrow guidelines is a bit developer centric and unrealistic.
I’d say WP Engine should work on their caching engine to allow PHP sessions and normal use of cookies. At a minimum WP Engine caching should be compatible out of the box with popular solutions like WooCommerce, MarketTheme, Cart66, MarketPress, Shopp, EasyCart and Tribulant.
It’s not an impossible technical order. Our preferred caching solution HyperCache does not suffer from these limitations (HyperCache is very flexible with mobile themes as well).
WP Engine is a good host and have some fantastic technology (staging areas, built-in caching, automated backups) but ecommerce support is an area where they could really improve.
Urgent Improvements to High Load Process
Even more so, WP Engine’s reaction to heavy load must improve. Media events happen unexpectedly and a business calibre host has to be ready to support your site instantly when they do.
What we do for our VIP hosted clients: we make sure there’s enough processor available, we cut back on any unnecessary processor intensive features (i.e.subscribe to comments), make some dynamic sections static and ride out the storm with our clients with next to zero downtime. With the advantage of scale, WP Engine can and should automate most of this routine.
Wyldsson Elite Nutrition should not have gone down for longer than five or ten minutes, the time it takes WP Engine’s tech team to automatically detect a 502, determine that it’s not DDOS (i.e. it’s legitimate traffic) and duplicate the site out to a higher availability node.
What happens afterwards and how WP Engine charges for the extra traffic is up to them. Taking down business sites during media events is not acceptable for premium hosting.
Alec Kinnear
Alec has been helping businesses succeed online since 2000. Alec is an SEM expert with a background in advertising, as a former Head of Television for Grey Moscow and Senior Television Producer for Bates, Saatchi and Saatchi Russia.
Hi Karol, this is Jason, founder and CTO of WP Engine. Thank you for the constructive feedback.
On the technical side, please note that we do support PHP sessions and all of the eCommerce plugins you suggest, it’s just that full-page caching is explicitly at odds with full page-caching.
This is also true of every other caching system. The plugin you suggest that is compatible — HyperCache — as well as other popular ones like W3 Total Cache, as well as server-based caching systems such as Varnish, Squid, or proxy-based caching systems such as CloudFlare — every single one of those systems also disables page-caching when PHP sessions or cookies are present.
So it’s not fair to say this is somehow unique to WP Engine.
Now, your criticism about how we should have proactively moved a site into another cluster during a traffic spike is, I think, well-taken. Building the ability for us to automate that — since spikes can come any time, and you don’t want to wait for human intervention — is really the solution needed. We also could do better in load-testing your site ahead of traffic, i.e. to be able to say “your site can handle N visitors per minute,” so that you could either think “whoa, that’s not nearly enough, we need to do something about that,” or “OK, that’s 1/100th of normal traffic so I’m OK even in a spike.”
So there’s definitely some tangible things we could do to address the concerns you bring up. Thanks again!
My advice is to steer clear of WP Engine. I do not run an eCommerce site, but WPE has been nothign but trouble. Their support is horrible. They claim their support members are “wordpress experts” in talking to one of them I discovered he did not know simple wordpress php functions.
I am currently trying to move to siteground. I have been waiting 24 hours on WP Engine to send me a full backup of my site so I can switch hosts.
The day I joined, I started getting 502 errors caused by corrupt cookies. I spent three days speaking to their support and all they did was blame my website. In the end I fixed the issue myself because they were no help at all.
Stay away from WP Engine.
Hi Jason,
Thanks for taking the time to address our concerns. I’m not quite sure what you meant when you wrote: ” full-page caching is explicitly at odds with full page-caching”. I’d be happy to correct that sentence for you if you post a change. We’ve been able to do partial caching for pages with PHP sessions and cookies using Hyper Cache. Our programming team will post more details on our caching methods. I’ll grant you, it wasn’t easy.
Hi Nick,
Thanks for sharing your experience. We have a business account of our own and have faced some issues ourselves. Some of our clients have had a similar experience to yours. WP Engine techs are very uneven. WP Engine is in a high growth cycle. In my opinion, WP Engine simply cannot hire senior engineers fast enough to cover all clients issues properly all the time. That said, WP Engine have some very gifted engineers on staff.
Why do we persist with WP Engine?
First, hosting is hard. Very few companies do it well. Second, WP Engine does have some lovely tools. The staging area, built-in caching, built-in backups, built-in CDN, managed upgrades and GIT deployment are all fantastic features straight out of the box. Setting these up yourself will cost you a lot of time and money. Some will be simply impossible to replicate on a microscale. WP Engine has some deals with Linode (core of WP Engine hosting) and other large providers which are simply not available to the public or even enterprise accounts.
WP Engine Downsides
I’m not really sure WP Engine is suited for the relative WordPress newbie as marketed.
Support
Some of the support responses rank with the most incompetent we’ve seen anywhere: just out and out lies or a very junior guy/gal shrugging you off. That’s okay for Foliovision: we know almost all of the time if an answer is correct or not and can persist until we get the right answer. But I’d hate to see a client face those duff answers.
Dubious Visit Counts
The per visit statistics are totally bogus and quickly can make WP Engine hosting some of the most expensive in the world for large, modest traffic sites. Why? WP Engine unlike everyone else in the world counts bots as human visitors, so if your site is well-indexed your site will generate bills for a site with very large traffic.
Again, in spite of those issues, we know how hard hosting is and recognise that WP Engine is providing a very good product at affordable (far from free but still affordable) rates. Our clients who have tried other dedicated WordPress hosting solutions like Websynthesis have had far worse experiences (sites down for a week at a time). The step up to real VIP WordPress hosting as we offer to existing clients is enormous: never a wrong answer, almost zero downtime. But it’s a totally different financial investment, as you pay for changes and fixes and support on a per demand basis. If you demand a lot from your hosting, you’ll rack up substantial invoices.
In your specific case, Nick, it looks like you were partly trying to offload programming work to your hosting company. It’s almost always a bad idea and any hosting company who wants to stay in business cannot afford to carry its customers programming load. I’m glad you’ve found an alternative host you like. Please let us know what you think about your new hosting in a month or two.
Hi Jason,
I’ve looked into our internal procedures more deeply. As you suggest, normally ecommerce requires turning off caching and increasing load.
When we use Hyper Cache, the whole page gets cached, no PHP is executed. I.e. we have the same situation as WP Engine: no cookies, no sessions. Like you on bog-standard ecommerce pages, we disable cache. In general we just disable it for pages that use a GET parameter. Of course, it can also be set to be disabled on specific URLs or based on specific cookies. Hyper Cache can first checks for these cookies.
Optionally, we use ecommerce plugins which respect page caching: the page is then pulled from cache and parts of the rendered page are built/replaced based on JS + Ajax calls. Often we custom build these solutions for performance.
For heavy parts of uncached pages, we use explicit/file caching: saving parts of the page to the disk and reading them from there, with a refresh rate set to anything that works (on one of our very busy WP Engine hosted sites we use 5 minutes for the menu). On a very heavy load site, we use memcache (caching into the memory), based on a unique key attributed to each part of the page. Another tool in the box is to check for a specific cookie/variable before deciding whether the visitor gets the cached version or a newly executed one. We are even able replace parts of the cached version based on such a cookie/variable. This however is a solution that has to be set for each client specifically based on their site’s functionality.
Our current advantage over WP Engine is that in case of heavy load our first response is not to shut down the offending site but to help the client through the traffic boom. Here’s some of what we might do in the case of heavy load:
So to recap what happened to Dave:
What could have happened when the load arrived is:
In this case, there would have been no downtime. The first two steps could be automated. For that matter, so could step three. The first human intervention necessary would be to react to Dave’s reply or to decide whether to turn off the client site in three days (most temporary spikes would be over by now) if there’s no reply from the client.
I hope these notes are of help for improving the WP Engine process during a traffic spike.
Making the web work for you, Alec
Alec,
I’m curious if you’ve any input on MediaTemple’s managed WP hosting as compared to WP Engine. I have been loving WP Engine since I began hosting with them last summer, but at the same price MT is giving you 10 million monthly visitors vs. 400,000 visits, 50 sites vs. 25, and 500GB storage vs. 30.
All of that is great, but primarily the traffic – I’m getting hit with $100s of dollars every month because of the success of my sites – is the reason I need to find another solution.
Unfortunately, bad experiences with MT’s Grid (I was a customer for 7 years, there were ups and downs but toward the end it just got horrible) leaves me wondering if that’s the right solution…
Hi Nathan,
I think you’ve answered your own question.
MediaTemple can promise whatever they like, but if they can’t deliver reliable hosting to back it up, high limits are no help. One of our clients almost lost her very high paying job, due to MediaTemple completing fumbling her chip company’s sites when there was an increase in traffic. MediaTemple was her choice, not ours.
Yes, WP Engine is pretty expensive. I’m none too thrilled that WP Engine is 1. charging a fortune 2. raising money to spend on marketing 3. not investing any raised money in improving the core product. Word of mouth is working great for Kinsta. From where I sit it looks to me like now WP Engine is a company being bulked quickly for a fast flip.
Not my style. I prefer to place my bets on companies who build a sustainable business by helping their clients succeed.
Was she on MT’s managed WordPress hosting or Grid?
Hello Nathan,
High end managed Cloud Servers. I’ve forgotten the exact terminology MT was using. In short, we had access to the best support MT offered at the time and it wasn’t nearly good enough. In fairness, WP Engine support does occasionally let us down (and we have access to their best technicians for most of our projects) with answers we simply know to be dead wrong. Fortunately escalation is usually swift and successful.
High end managed hosting is a losing proposition financially to your host, unless your host is at Rackspace or vip.wordpress.com pricing levels. There just are not that many affordable qualified server specialists. Systems can only take you so far without competent help. Wired Tree for example have great systems but most of their hands on support are complete numbskulls unable to swap a hard drive successfully.