Realtime WordPress Comments: Disqus vs wpDisquz vs Epoch

Comments

1. Jason from Postmatic 23 October 2015 at 4:35 pm

   Thanks for the review of Epoch! We’re about to push a nice update to it in the next few days which has a focus on performance improvements. You can follow along (or jump in) at github.com/postmatic/epoch/issues?q=is%3Aopen+is%3Aissue+milestone%3A1.0.8

   We have some interesting cacheing solutions in this release which should solve performance issues across the board. It’ll also fix the bug of Epoch loading itself when not needed (comments closed, etc).

   What this release will not solve, however, is performance on posts that have more than 100 or so comments having a long page render in some browsers. That one is going to require we get moving on rolling out pagination support and/or a js viewport that only loads the comments you need to see (think imgur galleries). That and a rebuild based on the REST api are queued up for 2.0.

   Reply
2. Alec 23 October 2015 at 4:45 pm

   Hi Jason,

   Thanks for the details on your ongoing development.

   The people who need Epoch most are publishers who enjoy hundreds of comments per post. The sooner the better for large scale performance improvements. As I mentioned, we have some comment caching code in Thoughtful Comments which you might want to have a look at.

   I’d put performance before features, considering the audience for Epoch (popular publishers). You might consider multiple configurations (shared hosting, small VPS) and dedicated server as they are very different environments.

   We look forward to seeing the new version!

   Reply
3. Jason from Postmatic 23 October 2015 at 5:01 pm

   With 1.0 we were focused on can it be done more than making a solution for large publishers. We were hoping to create something that is a good fit for small to medium sites and, more importantly, get some momentum and excitement moving.

   To that end I feel we have been successful: Epoch has turned into a mature open source project with contributions from across the WordPress world and a growing list of supporters. That’s more than I could have hoped for when we started out.

   Our 1.0.8 release will make it a good fit for even larger sites such as those that do get a ton of traffic but less than 100 or so comments per post. Up next in 2.0 will be solutions beyond that…..

   Best, Jason

   Reply
4. gVectors Team 27 October 2015 at 4:16 pm

   WoW!!! Thant’s perfect. We were waiting for such article and we got it. Thank you very much Martin! This is a really great research.

   I just want to say if you could wait for 3-4 days you’ll write totally different article. We’re happy to announce that in few days we’re releasing “wpDiscuz 3.0 Supercharged Comments” version. This is absolutely different plugin.

   1. It dozens times faster than current release,
   2. It use dozens times less memory,
   3. The live update is 500% optimized and it works dozens time fast,
   4. It supports any kind of caches and CDN,
   5. SQL requests are dozens times less,
   6. Lots of bugs are fixed,
   7. It comes with comment sorting options (newest, oldest, most voted),
   8. Individual links for comments,
   9. SEO improved,
   10. Super responsive on all kind of devices, different layouts on smartphones, tabs and desktop,
   11. Two type of pagination` ayax loader and simple pagination,
   12. CSS improved,
   13. 100% integrated with WordPress comment native settings,
   14. Individual comment subscription options,
   15. Lots of new settings ( min/max comment text length, share buttons… ),
   16. And lots of more, even core objects and classes are optimized.

   Please wait for a few days and I hope you’ll get tons of fantastic arguments to change or write another article abut WP Comments plugins.

   — Thank you wpDiscuz Developers gVectors Team

   Reply
5. Alec 28 October 2015 at 3:33 am

   Hi wpDisquz people,

   Great to hear.

   We could definitely use those new features and have wpDisquz in play on some larger and very active client sites. Please tell us when these features are launched so we can test them and send you some $$ support.

   Making the web work for you, Alec

   PS. Could you be kind enough to sign your comments in the future? It’s so much easier to talk to a person than “a team”. Trying to seem like Microsoft as smaller boutique companies plays against our strengths (the personal).

   Reply
6. Tom 28 October 2015 at 10:57 am

   Hi Alec, Sure… I’m Tom. And thank you so much for your interest in wpDiscuz plugin and for your willingness to support it. We’re currently testing the 3.0 version and doing our best to release it by end of this week. We’re testing wpDiscuz with 10-50k comments on each post, the parent comments have 5-10 dept replies. The native WordPress comment system in most cases can’t work even with 1000 comments. If it doesn’t stop loading it loads those within dozens minutes. wpDiscuz 3.0 really supercharges the native comment system and it removes all limits related to comment counts and website high activity…

   Reply
7. Jason from Postmatic 28 October 2015 at 3:15 pm

   Hey wpDiscuz,

   Glad to see you all keeping up the good fight. We’re big fans of what you are doing and love watching the evolution of your plugin. Native commenting FTW!

   Jason

   Reply
8. Alec 29 October 2015 at 10:29 pm

   Hi Tom,

   That sounds wonderful. Let us know as soon as we can download and help with the stress testing. I promise you very detailed feedback.

   Hi Jason,

   We’re looking forward to the next revision of Epoch too. For these sites though we need to be able handle high comment counts.

   Thanks!

   Reply
9. Tom 30 October 2015 at 9:08 am

   Thank you Jason and Alec, We relay appreciate your kind attitude and support. We keep supporting Postmatic in wpDiscuz 3 and relay like it. I see the reason of creating Epoch, and it’s very good developed, but like wpDiscuz it has lots of things which should be made better. So keep updating and lets bring people from 3rd party comment systems to native WordPress. The wpDiscuz 2 has lots of very useful features and it faster than 3rd party comment systems, but the performance and CDN compatibility was not good. So the main advantages of the new wpDiscuz 3 release are performance, speed on 100 and more comments (the real speed diff you can see on 100, 1000 or more comments), cache/cdn, live update optimization and super responsiveness on diff devices. i’ll be in touch very soon. It seems everything is going well with testing…

   Reply
10. Tom 2 November 2015 at 10:49 am

    Hi, just want to let you know that wpDiscuz 3 is released. We’re on very active supporting stage… Detailed changelog you can find here: wpdiscuz.com/wpdiscuz-3-0-0/

    Thank you!

    Reply
11. Jason from Postmatic 2 November 2015 at 2:59 pm

    Hey Tom,

    Thanks! And, you’re welcome. We think wpDiscuz is great as well…. and agree that yes, there is room for all of us in this space. We see wpDiscuz as a great fit for sites that want an all-in-one native commenting solution, and then have Epoch for folks that want things to be more piecemeal. Let’s keep at it and get comments back where they belong.

    Keep up the great work! Jason

    Reply
12. Martin 6 November 2015 at 5:32 pm

    Hello Tom,

    we just added a part about wpDiscuz 3: foliovision.com/2015/10/disqus-wpdisquz-vs-epoch#wpdiscuz-3

    Thanks, Martin

    Reply
13. Tom 6 November 2015 at 6:43 pm

    Hi Martin, Thank you for testing it again, it seems you’ve tested the auto-optimizer SQLs. It Works only one time on first page load. So please make sure this are correct data, it’s real important, and make sure you’ve tested 3.0.3 version.

    And it seems you’re not see what’s happening here:

    1. wpDiscuz 3 can’t be slower than wpDiscuz 2. That’s fisicaly impossible.
    2. wpDiscuz 2 and the native comment system loads all comments using one SQL so it overload website memory and stops loading on active websites.
    3. wpDiscuz 3 instead of overloading memory it get’s very small partial data with very light SQLs. even 2000 SQLs of wpDiscuz 3 are dozens times faster than 42 SQLs of wpDiscuz 2 and native comments for 1000 comments. I’m repeating, for 1000 comments, and for 10,000 comments. This kind of working allows to load and very fast perform thousands of comments. You should see this aspect. You’re only tracking SQL numbers without profiling script ability.

    Here are more info about SQLs: wpdiscuz.com/wpdiscuz-3-0-x-sql-queries/

    And here is an example how wpDIscuz 3 works very fast on website with 3.5+ million comments: wordpress.org/support/topic/suggest-ondemand-comments-load#post-7630702

    Reply
14. Tom 6 November 2015 at 9:17 pm

    Ok, I see the issue. You’ve tested wpDiscuz 3 without optimizing comments. So in other word you’ve not tested wpDiscuz 3. Please do another one or just correct the text in post. wpDiscuz 3 caches full parent/child relationship on first load. it may be x2 slower and with thousands of SQLs. BUT this is only on first time, that’s it. So you should check the second load of comments and you’ll see the correct result. You’re currently talking about comment optimization process SQLs and Speed.

    You’re saying:

    the plugin tries to cache this into wp_commentmeta but it must be failing for us. I guess if this works for you, it can be faster.

    What’s the mean to report wpDiscuz 3 testing result without using the wpDiscuz 3 comment caching/optimizing system? That’s obviose that you’ll get such a bad result….

    1. If you’re testing 3.0.0 – please make sure [Supercharge My Comments] button has done its work 100%
    2. If you’re testing wpDiscuz 3.0.3 > – Make sure you’re not testing first load of comments. All 1000 comments should be loaded one time, before testing.

    Hope you see what I mean and you’ll change those result, because those are not correct for wpDiscuz 3.

    Thank you for you time.

    Reply
15. Alec 6 November 2015 at 11:35 pm

    Hi Tom,

    Thanks for the testing notes. We’ll be sure to add your additional steps next week. I’d suggest making the process easier for your users with clearer explanations.

    Thanks!

    Reply
16. Tom 7 November 2015 at 8:06 am

    And some good news. wpDiscuz 3.0.4 is released with comments Lazy Load function. So it loads comments when you scroll page to down. That way you only load the post header and content. The comment section doesn’t use time and memory. It boosts performance delaying loading of comments in long web pages because comments outside of viewport (visible part of web page) won’t be loaded until the user scrolls to them. This reduce memory usage, page loading time and SQL queries. On a page load wpDiscuz only runs plugin settings initiator SQLs, those are about 5.

    Reply
17. Alec 7 November 2015 at 2:30 pm

    Hi Tom,

    Thanks for the update.

    I’m quite worried about the Lazy Load function. Comments play a significant role in Google rankings (we’ve tested for some of our clients). I think they in particular trigger the recency bonus. If wpDisquz 3 stops Google bot from reaching and indexing all comments, you may be doing your publishers’ sites a great disservice.

    A more search friendly approach would be not to delay the loading of the body of the page yet load the comments later regardless of scrolling.

    Reply
18. Tom 7 November 2015 at 6:05 pm

    Hi Alec, Sure, we know about that, and we use AJAX Crawling Technology for our ajax loaded contents, this works for Lazy Load as well. If you inspect page source you can find such link: http://example.com/#!loadmore&parentId= xxx

    More info here: support.google.com/webmasters/answer/174992?hl=en

    So you should not worry about comment indexing issue.

    Reply
19. Alec 7 November 2015 at 7:02 pm

    Good to hear Tom! Thanks for the interesting reference. Have you field tested this (searching for exact phrases which appear way down the page)?

    Reply
20. Tom 9 November 2015 at 10:49 am

    The AJAX Crawling URLs are the same for Lazy Load and for other load more buttons. We’re releasing 3.0.5 within next 24 hours with stable SEO features: Here an example how wpDiscuz loads ajax contents for Search Engine bots (this is the same for Lazy Load):

    1. I show #52 comment
    2. in firebug [load more] button with /#!parentId=51 SEO fragment
    3. parsing logic from /#!parentId=51 to ?_escapedfragment=parentId=51
    4. I call this URL and it loads #51 comment
    5. Testing on Fetch as Google Tool and it loads correct comment

    Video: screencast.com/t/CXrrTIpCiE3D

    Reply
21. Martin 9 November 2015 at 3:35 pm

    Hello Tom,

    just a quick note before I get to check this in detail –

    I was testing wpDiscuz 3.0.3. I tried to reload the same post over and over again, but I was getting no speed improvement and the SQL count didn’t lowered. Perhaps the built-in SQL optimization didn’t work for us.

    Thanks, Martin

    Reply
22. Tom 9 November 2015 at 5:37 pm

    Hi Martin, Thank you for your time and the hard work you do testing wpDiscuz. We really appriciate it. Here some info that should used to see the correct advantage of wpDiscuz 3:

    The main idea you should know that, native WP and wpDiscuz 2 get all 1000 comments using one SQl, if your post has 10,000 comments it’ll try to get it using a one SELECT SQL. So the number of SQL-s are 1 instead of 1000 or 10,000. This logic doesn’t work if you have many comments. it’ll stop loading comments. The memory of hosting service can’t store thousands comment on one page load and the MySQL buffer limits will not allow you get data. It’s just a matter of fortune that the native WP and wpDiscuz 2 on your server can load 1000 comments. Believe me about 90% cases WP and wpDiscuz 2 stop loading. And you can see that 1 SQL is hundreds of times can be heavy than small 2000 SQLs. So I recommend don’t report wpDIscuz 3 performance based on SQLs.

    wpDiscuz 3 doesn’t use this logic, it only gets parent comments of current displayed comment threads and use very light and fast SQL to get hierarchy data. This works for each comment. There may be 1-3 SQLs per comment, it depends on enabled features. This SQL takes 0.0001-0.0003 second. 1000 x 0.0003 = 0.3 sec for 1000 comments. Please see this screenshot: screencast.com/t/wZgxgPj6

    And ATTENTION!!!) It doesn’t matter how many comments you have for certain post. WP native system SQL get ALL comments, even if it shows 1 comment on front-end (paginated). So if you have 10,000 comments for one post, and you show 10 comments on first load, wpDiscuz 3 spends 0.0003 x 10 (SQL) = 0.003 seconds. But WP Native comment system will use the same resource (mysql time and memory): screencast.com/t/kdXYzn63X about 0.1 sec.

    Nobody set 10,000 comment at once. The maximum number of loaded comments can be 100. So wpDiscuz 3 with 0.03 sec query time will always save your server resources. but the native WP will always operate 10,000 comments in DB. Even I’m not talking about big difference of memory usage.

    P.S.: Please, I’d like to ask you to track SQLs like I mentioned above. Do not show SQL numbers, those show reversed result. Those don’t show the correct advantage of wpDiscuz 3 over native comment system and v2.

    Reply
23. Martin 19 November 2015 at 4:10 pm

    Hello Tom,

    I tried that again with version 3.0.7 and no luck either. I logged all the SQL queries, but I don’t see any inserts of wpdiscuz_child_ids into wp_commentmeta. We tested this on WP Engine host in staging environment. I did nothing to break the “comment caching/optimizing system”.

    We have some other website where we are using wpDiscuz 3 for around a week and I can see some wpdiscuz_child_ids on that one – so perhaps it works there. However the post with the most comments is just as slow as it was in wpDiscuz 2.

    I checked your video with console as well. In our case loading 500 comments via Ajax takes on average 6.3 seconds, which is at least twice as slow as your loading of 900 comments in 4 seconds. I’m only checking the SQL query counts to understand why the load times are not improved.

    Don’t get me wrong, I would like this to work and we really need it to work as the website where we use this will be getting the most traffic (10x as much) in December, so we really need something fast.

    I also tested the Lazy loading of comments and I think this is the part which improved the most together with the “Load more comments” performance. I tried setting it to 5 parent comments per page and then it worked fine – the commenters here use a lot of nested replies. So clicking “Load More Comments” button typically loads 100 more comments in 2 seconds.

    Thanks, Martin

    Reply
24. gVectors Team 30 November 2015 at 3:57 pm

    Thank you Martin, I’ve posted comment a week ago here. But for some reason it’s not published. I just want to say thank you for your time. And ask you to change this post content related to wpDiscuz. Because that’s not actual for current 3.0.9 version. Since wpDiscuz 3.0.8, all SQLs have been totally optimized. And its even more fast and light. It comes with 39 SQLs instead of 2000 like it mentioned here. And the 39 are for unlimited number of comments.

    Reply
25. Martin 11 December 2015 at 10:32 am

    Hello Tom,

    sorry about the delay, I was meaning to get back to you much earlier. I’m not sure what could happen to your comment, we got no notification.

    We tested wpDiscuz 3.0.9 on a clean Linode with a clean WordPress and no plugins. We only imported a single post with 801 comments and associated user account to test on.

    Here we compared the resulting HTML page size, number of SQL queries, PHP processing time and maximum memory usage:

    100 comments per page (we are talking about actual number of comments that shows up, as the WordPress setting determines the number of top level comments rather than number of comment that actually show up, so we tweaked that setting until we were getting 100 comments on our test page)

    • no plugin:
      • size: 190 kB
      • queries: 174
      • time: 0.55 – 0.64 s
      • memory 9 MB
    • wpDiscuz:
      • size: 320 kB
      • queries: 203
      • time: 0.65 – 0.76 s
      • memory 10 MB

    Reply
26. Tom 12 December 2015 at 9:18 pm

    Great! Thank you very much Martin. As far as I see the huge change in the number of SQLs is already in testing result. The only reason why we don’t see the real speed is the cache. We’ve made some changes to make it more compatible with caching systems. And it’s already available in 3.1.0 > versions.

    Also we should not forget that wpDiscuz has +40 features. And it’s not x40 times slower than the native. So you can see how fast is the core that it can load +40 features with approximately the same speed as the native. However It should already even be faster.

    Reply
27. Oliver Liu 30 March 2016 at 5:44 pm

    Hi Martin and Alec, thank you very much for the article with details and the benchmark data, it is very helpful. I am a developer working on a travel/tourist blog website preparing to process big data. I was looking for a chat solution and your articles really inspired me. I have made some changes to Epoch to interact with dynamic html code efficiently generated from mysql directly, in the wrapped template expected by Epoch. With the help of some custom fields and custom built index, the overall performance has been greatly improved. Here is a demo page to test the loading speed from over 150k comments nested under the same post (which are bible translations in public domain for users to quote without copyright concerns, each verse is one comment). There is a slight delay in displaying 1,000 comments per page, but 100-200 comments per page can be handled without difficulty. The comments are loaded at the same time of generating the web page, not through ajax post method, and thus is easy to cache such as using browser cache.

    tagnmap.com/wp/photo/the-holy-bible/ Just append ?limit=200 (number of root comments) to the url if you would like to test how it performs. The comments (root and children) are kept at 1000 per page.

    The website is currently hosted on Amazon t2 micro (VPS with 1gb memory), and this solution works on VPS and above. The comments are generated dynamically with user custom information (voted comments, bookmarks, likes, etc) , and is close to real time chat with user dynamics.

    I believe with some fine tuning the solution can be a candidate for busy commercial websites. Please get in touch if some of your clients might be interested.

    Hi Jason, not sure you will ever come back to this post. Nice work on the Epoch framework. Do you have plan of making a premium version of Epoch? I will try to get in touch with you to catch up.

    best regards.

    Oliver

    Reply
28. Alec 1 April 2016 at 10:25 am

    Hi Oliver,

    Your real time chat commenting project is fascinating. If you can keep the load down both on the client side and the server side. Even Disqus offers execrable performance. We ended up going a different direction and built more caching and like/dislike buttons into our Thoughtful Comments which now loads like lightning (20 ms) even with hundreds of comments.

    Performance oriented websites should be looking more at comment caching than comment features which for the most part are lipstick on a site. I know there are exceptions and I’m not underestimating the value of community but fast loading pages (with lots of comments) build an audience and a community too.

    Please post another progress report when you are ready though. Once it’s working well enough for commercial deployment, we definitely have a client who would be interested.

    Reply
29. Oliver Liu 4 April 2016 at 7:51 am

    Hi Alec,

    Thanks for the prompt feedback. You have made an excellent point in caching comments and reducing workload. After posting the previous comment, I studied your other article on comment caching and started to make further improvements to my solution by generating and storing caching pages at the server side, and have made some good progresses.

    As for the storage of comment cache pages, I took an approach different from Thoughtful Comments in that the cached comments are stored as a custom post type, and are divided into separate posts if there are too many comments to show in one page (such as the bible verse demo page). This is mainly out of consideration for integration with indexing, updating comment cache in distributed computing environment and load balancing if the client’s site gets really popular.

    Currently there are two cache versions, one for non-logged in users, and one for logged in users who have not commented or voted on the post. The difference is logged in users will have the links to reply comments and vote (thumbs-up) and can participate in the discussion whenever they like. when a post is requested, the script will first check whether the requester has logged in and has commented/voted on the post, if not, then the server will serve the post content with cached comments. The cached comments (which is actually stored as a separate post) are treated the same way as the post content, and can be cached on server side and client side. If the custom post of cached comments does not exist, it will be created on the fly saved the content in db for subsequent visits, and served to the requester. the cached comments are already in html format ready for browsers to display without parsing.

    The total number of approved comments and mapping of search query to cached comment post are kept in memory, and the cached comment post will be updated at preset interval if the total number of approved comments have changed. If the post has closed for comments, the cached comments post will remain the same, saving valuable resources for logged in users active on the current post.

    On my test set up, it takes less than 1 ms to find out the ID of the custom post containing the cached comments directly from memory without hitting the database or harddisk, and less than 1 ms to retrieve the post from WordPress db cache. so this should help to reduce page loading time. This set up uses some db or disk storage space, and some memory to hold the id/query variable mapping, and CPU usage is quite low.

    New comments are loaded via ajax request as Epoch team originally designed.

    It is relatively easy to build additional filters to show specific comments, in different views (timeline or nested tree), or comments within the last hour/day, or pending comments awaiting approval. I have built two filters for the demo page (bible translation and verse number, so when users click on the link, it will show the requested version or verse only, without leaving the page. The filters are processed using values already in memory without querying the db, and is very efficient too.

    Currently the solution is customised for my project and hardcoded for efficiency. If there is a real need for this solution, I can separate the code from the project and create a premium WordPress plugin. I have sent an email to Epoch team to understand their development focus and direction.

    The test server was working fine on Saturday, but since Sunday there were some connectivity issues which may have been caused by an unfinished cron job on the VPS, or interruption to Amazon services in general. When the server could be reached, sometimes the page with cached comments could be loaded between 3s to 4s displaying 1000 comments per page, while a few other plugins ( WooCommerce, JetPack, Pods) are running at the same time. I will do some further testing and report back the progress after hearing from Epoch team.

    The comments with user information and indexing stay in the website which is an advantage over third party commenting solutions.

    Thanks again for sharing your knowledge on comment caching, and tips on performance.

    best regards .

    Oliver

    Reply
30. Jason Lemieux 19 April 2016 at 4:41 pm

    That’s a wickedly cool demo, Oliver. I’ll follow up with you via Github. Epoch is alive and well… twitter.com/gopostmatic/status/722422956748615680

    Reply
31. Steve Wilkinson 20 March 2017 at 12:18 am

    Very interesting discussion, especially in the comments. I just wanted to not that when you said you ran the tests on WP Engine staging server, that AFAIK, WP Engine staging servers don’t cache.

    Also, as I don’t deal with high-volume comment situations (yet), the main issue is UI/UX, not performance. Disqus, IMO, provides a pretty good overall user-experience. That said, I’d much rather stay on-site/native, so if wpDiscuz ends up close enough in the OVERALL experience to Disqus, you’ve probably won me over. (I’m hoping to test it on one of my sites in the next week or so.)

    Reply
32. Alec Kinnear 19 May 2017 at 2:35 am

    Hi Steve.

    Our own Thoughtful Comments will end up being a Disqus replacement with better performance, a nicer look (very close to Disqus) and an easier to use interface. If you’d like to beta the pro edition, please drop me a line.

    Reply