Building a better WordPress Post Editor

There’s been a lot of talk and writing about radically revamping WordPress edit experience since the New Year. It’s great that the conversation has been started and many strong ideas have been shared. Matt Mullenweg kicked off the discussion on 4 January with a Coleridgian description of a new editor. Like Kubla Khan’s “stately pleasure-dome”, the new editor should be a “miracle of rare device”.

The editor will endeavour to create a new page and post building experience that makes writing rich posts effortless, and has “blocks” to make it easy what today might take shortcodes, custom HTML, or “mystery meat” embed discovery.

Any “post building experience” which aspires to make “writing rich posts effortless” has my full attention. What should this experience look like in practical terms?

Enthusiastic talk is focused on ContentBlocks. ContentBlocks are kind of widget to allow an author to post different kinds of content from text to video to audio loops to slideshows. The html content box will be obsolete and instead writers will assemble different ContentBlocks together to create a string of content which becomes a post or page.

The current editor content window would become more or less obsolete. Like Russian dolls, ContentBlocks would be designed to house other content blocks infinitely. 2017 Customizer lead Mel Choyce writes:

I think many of them [ContentBlocks] will have to be nest-able, especially once we get into the customization and layout side of things.

As well as removing, a conventional editor from the edit post interface, there’s even talk of moving editing as a primary function into the Customizer. The Customizer is a kind of site builder tool into which most of WordPress’s development resources go and is being slowly built-up as an alternate admin interface. The goals seem strange with priorities like “build your whole site from your phone” or “move any piece of content on your site to a different part of your site, just by dragging and dropping it where you want”. There would be little left of the traditional edit experience. Writing in WordPress would become like Tom Cruise’s investigation software in Minority Report. Without structure, you can do anything, anywhere.

Structureless editing seems fantastic (in the original sense of the word).

Futurology is an invigorating and refreshing process. While there is nothing wrong with talking about ideas, pulling technology from too far away in the future before the underlying technology is ready results in user and commercial failures. A clear example is the pocket organiser which failed miserably for decades before it succeeded: Newton, Palm Pilot and even Psion’s Symbian. The underlying ideas of a pocket work tool did not become useful until the iPhone at which point its forerunners were already obsolete:

In 2006, Symbian had 73% of the smartphone market,compared with 22.1% of the market in the second quarter of 2011.

In the end, Symbian enjoyed few of the fruits of Psion’s endless toil. iOS and even later arrival Android took the whole market.

What is particularly dangerous about this wild talk about re-imagining editing is WordPress co-creator Matt Mullenweg, plans to spend most of the resources of the WordPress.org on this grail quest over 2017. Time lost running after chimera will be lost and a year’s opportunity to make WordPress a more powerful and useful publishing tool will be lost.

Worse yet, if publishers are not using a tool like our BusinessPress plugin to stay on security only updates, the growing pains of a new editor will cripple a quarter of online publishers (current WordPress CMS marketshare). Wild whimsy with what is a production tool means tens of billions of dollars of productivity losses. Until the new editing paradigm is production ready and battle tested, it has no place in a release edition of WordPress. WordPress publishers would be better served by focused improvement of the current substandard TinyMCE edit experience “notorious for stripping out markup“.

Later, I’ll cover my concerns with ContentBlocks in more depth. I’ll first look at why so many people are unhappy with the current WordPress editor and editing experience. Next I’ll cover how an ideal WordPress editor might look like and how it would work. The right new editor could improve workflow and enhance the current experience without requiring retraining of editors nor rewriting tens of thousands of plugins and/or themes.

What’s wrong with the WordPress Editor

We built a WordPress WYSIWYG in 2007. Starting in 2015, we tried to retire our Foliopress WYSIWYG for two years. Unsuccessfully. We had to bring Foliopress WYSIWYG back from limbo. This principally involved fully integrating the WordPress Media Library. In praise of the WordPress, the Media Library has grown to be very powerful and easy to use. It’s awesome.

What are the issues with WordPress content editing which drive us to maintain a separate WYSWIYG editor? Code and javascript mangling. The WordPress TinyMCE editor does not reliably maintain paragraphs and linebreaks. It’s a bit WhatYouSeeYouMightNotGet. When creating complex nested tables and code examples our editor never fails (don’t thank us, thank FCK in Poland, we just maintain a solid WordPress port).

We’ve done what we can to improve WYSIWYG by in our editor by allowing with just the tiniest modification of your theme CSS real WYSIWYG. This only covers the CSS though and not the javascript or the short codes. What would make the WordPress editing experience amazing would be a live preview tab which turns the shortcodes and media into their visuals.

What is missing in WordPress editing with the shortcodes and code inside a fairly utilitarian textbox is true WYSIWYG. The only way to preview a post properly is to open it up with “View Post” and then reload the page to see any changes.

This is slow and clunky.

The overall editing feel in WordPress follows that trend. Already in 2013, Torquemag compared writing in WordPress with writing in Editorially (since acquired by Vox and removed from the market):

Meanwhile, the WordPress editor, which was my friend for so long, already seems like a distant drudgery. I’m probably being a bit overly dramatic when I say that. But again, it’s the tiny release of friction like not having to stop my thought to italicize a word, repeated many times a day, that makes the tiny feel large.

When we looked around at a new editor to build on for WordPress last year (1) we settled on redactor.js. In 2012, WordPress TinyMCE maintainer Andrew Oz was very sceptical about Redactor’s or any other JS editors suitability for WordPress:

I like it too. However it’s far from being ready for anything more than comments/bbPress editor….no plugins, no way to extend it other than hack into it, very minimal APIs, try copying and pasting something from another browser tab, then look at the HTML. And a big one: no undo/redo for all actions?….Roughly it would take a good team of several full-time developers at least a year to bring it closer to TinyMCE.

Nearly all JS editors get off the ground fast by adding UI to the browsers built-in contentEditable functionality. Once there, most try to tackle few more tasks and stop. The next step requires a lot of work: creating APIs, dealing with tons of browser quirks, etc.

Currently there are several other editors at approximately the same stage. The problem is that a lot of the functionality for editing HTML should be in the browsers. It can be handles much better there than trying to extend the limited contentEditable mode with JS.

So unless the browser vendors decide to sit down and make and implement a nice specification for editing of HTML, don’t see how this situation would change.

We are a long way from 2012. Browsers have improved their support for live js editing within the browser. Redactor is awesome visually, very fast, beautiful preview, lightweight fast editing tools for media. In short everything we’d like the WordPress editing experience to be. TinyMCE will never have that shiny and fast and brilliant feel. It’s in neither the personality or the technology of the software.

As a baseline, the new editor experience for WordPress should not be worse than the Redactor’s demo on Imperavi’s home page. It should look great and have zero lag. The fast track here would be for Automattic to buy Redactor outright and open source the WordPress platform version.

The Ideal Editing Experience 2017

Starting from the beautiful and spritely Redactor demo, it’s easy to imagine a much joyful WordPress editing experience.

The ideal WordPress editor would have three panes:

• Visual Preview (with optionally some basic front end edit text tweaking on text sections)
• Writing (Markdown with both Content Block edit and/or shortcode)
• Code (raw HTML, shortcodes; displayed with coloured syntax for readability)

Why three different panes? To please all users.

• Pro writers want to write in Markdown.
• Beginners want WYSIWYG (which they get from the Visual Preview)
• No one wants to edit in code but code is damn handy when you need to diagnose serious issues or when your WYSIWYG suffers hiccups not correctable in its markup syntax.

What’s annoying about editing with WordPress now:

• how long it takes for the front end editor to preview in a new post
• how much damage TinyMCE does do line-breaks and paragraphing (TinyMCE must go)
• how thoroughly useless the code view is (no colour coding)

Hence to improve the WordPress editor experience, we need:

• amazing and instant preview (js)
• wonderful background conversion library (Pandoc converted to js)
• robust code editing environment with colour coding

Three instantly switching panes would solve the issue. Technology has caught up to make this kind of editing power available on most devices (a fallback version without instant preview and without coloured code highlighting could remain available for weaker devices or better yet app style editors could be created for the mobile devices).

Limited Visual preview exists for shortcodes in TinyMCE but what does exist is tantalising

Creating instant preview, live converting between html and Markdown, coloured syntax highlighting would be a huge amount of coding under normal circumstances. Fortunately, as WordPress is a fully GPL project, we don’t need to start from scratch.

Many wonderful libraries of open source are available to us for both the code editing environment and for the conversion library (Pandoc would have to be converted to JS (alternate technology) to be used in real time). There are great libraries for coloured code editing, such as Ace (previously Bespin) from Mozilla. With more work a more traditional code highlighter (SHJS or SyntaxHighlighter or Prettify or highlight.js) could be extended to work with live editing.

WordPress developers would start from the best existing libraries and build bridges between them (2).

Writing mode vs Instant Preview

Most people would spend their time working in Writing mode and almost never need to go to code mode. When in Writing mode, one can write intuitively at speed. There could be an option to have put Writing and Preview mode in side by side view where there is space (or even Preview and Code Mode although working on that aspect is really secondary/non-essential as few people would use it).

This technology is here now. There are any number of editors on OS X providing live preview and Markdown right now. Two of the best are IA Writer and Mou in which I’ve composed this article, including dropping in pictures.

How to create Fluent Workflow

Speed is of the essence. That means better and more intuitive workflow. Even with a much better editor or content blocks, it’s very tough for writers to quickly replicate publishers’ complex post formats. Extensive training for writers to create the content properly and then editors’ time working to make the stories match format properly is a huge waste of valuable resources. There must be a better way to create simple templates for database like content or visually complex magazine style content.

Custom fields already exist for custom content templates. Yet custom fields remain awkward and unintuitive in their raw form, sitting in a lump under the post edit window and excerpt. A better alternative would be to encourage developers to build Custom Post Types which are primarily Custom Edit Layouts.

We’ve just finished three such custom posting interfaces.

Five years ago we built an advanced and powerful plugin called FV Testimonials, using Custom Post Types as the core. In its functionality there were as a testimonial, bios and a portfolio publishing system. FV Testimonials used the custom post type interface in a fairly raw form and used sophisticated categorisation to keep the three genres separate.

There were three primary problems with the all-in-one system of FV Testimonials:

• lack of intuitive function when using the interface. Even though I’d had a role in speccing the software, I had to look at existing content to be able to use it myself.
• lack of genre specific terminology. It’s much easier to work quickly when a bios edit interface specifically discusses name bio and headshot than when it uses generic terms like “Title”, “Short Description” and “Image”. The cognitive load of visualising the generic terms as genre specific words disappears, freeing the mind to focus on the work instead.
• lack of specific workflow. The main description block came much to high in priority.

When we divided the master plugin to separate plugins we were able to:

1. use precise terminology for each workflow.
2. move the edit fields in the primary posting interface around to follow the standard thought sequence when creating or editing a bio, a testimonial or a portfolio.
3. create additional management screens to suit the task at hand

With all of these in place, creating bios or testimonials or portfolio items is a pleasure rather than a pain. How does this affect core WordPress? Core developers could study this process to make it easier for developers to create edit templates instead of adding free form content blocks which will contribute confusion rather than fluent workflow. Larger publishers have specific templates and which are best served by templates. Let’s make it easier for them and even for smaller publishers to create post templates which respect their industry and process.

Publishers already enjoy free form editing on the side of anarchy. Publishers do not need more flexibility, publishers need smoother workflow.

How to handle Content Fragments/Sniplets/Snippets

What doesn’t fit well into the WordPress (or any other javascript editor)? Blocks of PHP and complex html blocks. While content blocks sounds like a solution to this issue, a better and more robust solution has existed in WordPress for more than a decade as plugins: simple snippet library technology.

Shortcodes are great but require coding full plugins which is beyond the reach of normal, even professional users and even beyond the reach of casual developers. On the other hand, preparing a useful PHP snippet or some standard boilerplate is not. A built-in library of snippets would be intuitive, powerful, flexible and intuitive.

One of the arguments against snippet library technology has always been that snippets and shortcodes are obscure in the text editor. When one adds real time preview technology (which TinyMCE simply doesn’t have), a full fledged snippet library built into WordPress could participate in preview. Those snippets of course would not be editable in preview mode but the surrounding text could be.

With snippets participating in preview, one of the great holes in complex publishing is filled with flexibility for publishers at all levels at once. Marvellously, there are at least two great plugins nearly ready with just a bit of polish for integration into core: Shea Bunge’s Code Snippets and Johan Steen’s Post Snippets, Both are heirs to the admirable but abandoned Sniplets by John Godley (who recently developed a Lightroom to WordPress plugin for Automattic).

Dangers of Custom Content Blocks

A separate argument is whether we really need content blocks (a kind of WordPress Widget thrown into the content editor). At one level, this appeals to me. I like a precise tool for each job. Yet I have several concerns about content blocks.

First, clutter. The idea sounds nice when there are four built-in content blocks plus two from your most important content-plugins. What will happen instead is that every theme will introduce four custom content blocks, every plugin will rush to add two or three of their own. The custom content block will fracture the edit experience like a modern day editing Tower of Babylon.

Plugin developers and WordCamp sponsors like Yoast have been allowed to take over and destroy every aspect of the WordPress admin interface, from the warning bar, to the update screen, to preference screens. Adding another attack vector from overzealous commercialisers is simply a bad-idea. Starting from the noble idea of making editing in WordPress easier, free-for-all will make editing with WordPress an absolute nightmare.

The idea of content blocks came from Medium. Many, including Matt Mullenweg and David Heinemeier Hansson, feel that Medium’s publishing workflow stole WordPress’s thunder in 2016. The WordPress crew who are running after Medium here should remember that Medium is a monolithic, locked down platform. Medium can add innovation like content blocks as the environment is controlled. We’re still working on getting control of the WordPress admin environment back from commercial developers for busy publishers. To remain a usable tool, WordPress must be very careful what opportunities we allow developers to control or interfere with the interface.

Second, workflow. How is it possible to develop a workflow and/or teach new writers to use your WordPress install when there are dozens of content blocks with which to wrestle? Simple content templates would reduce training and radically increase success rates for less technologically proficient writers.

Third, fragility. This kind of fiddly code is ripe for conflicts between versions of WordPress and installs. A bad content block could leave you with huge patches of content not working at all or an inability to migrate content from one site to another. More on that below.

Fourth, lock-in. WordPress already practices far too much lock-in keeping media content in paths like /wp/wp-content/media/. Media content should be in top-level directories like /images/ and /video/ (although video usually should be hosted offsite with limited exceptions for small quantities fronted by a CDN). Content blocks threatens to make moving between themes or changing plugins or working on different sites will a huge burden. As open source and open web advocates, we should instead be building WordPress to be portable and to lead into other parallel solutions and/or future solutions. Any attempt to privatise the web should be anathema to us (3).

An improved interface for Shortcodes would be welcome (there is a featured plugin project called Shortcake who is part way down this path). But the robust and unbreakable and backwards compatible Shortcodes should remain as the underlying technology in code view.

Conclusion

Ironically, my project against WordPress lock-in could do the most to rejuvenate WordPress and save its longterm marketshare. It is essential that technology remain compatible with the rest of the CMS world if WordPress wishes to continue to attract the platform agnostic publishing world. A revolutionary content block based free form workflow is more likely to alienate a huge proportion of publishers than attract new ones.

Workflow, fluency of editing and ease of use are the three values required in an editor. If we build that editor and then tell the world, it will be a very good reason to use WordPress. Marketshare is not a persuasive reason to use a product. Cool factor not backed by performance is equally unpersuasive.

If we want serious publishers to continue to use WordPress we must improve their lives. If WordPress loses the serious publishers, in the long run WordPress will lose its mindshare and with that its marketshare.

---

(1) FCKeditor has since been retired. FCK’s successor CKeditor is roughly the same experience better behaved on mobile. Not different enough to be worth the hundreds of hours needed to adapt CKedit properly to WordPress.

(2) We’re more than happy to put our own shoulders into the project of building a shared editor. A WYSIWYG editor is too much for a single company (we built one already and our major competitors have all abandoned their own) and offers very little ROI (historically people won’t pay for an alternative WordPress editor). At the same time, the editor is such an essential experience that it does make sense as a group to invest in its improvement.

(3) It’s different for Automattic who have taken VC and have their arms manacled but those of us who have really built WordPress did not build WordPress to build marketshare for a specific CMS or web app but to contribute to a publishing tool we could share with others until something better comes along. Many people at Automattic worked independently on WordPress before being hired to Automattic so in spirit, there are many, many Automatticians who ardently believe in free software. When I refer to Automattic here, I’m speaking of Automattic’s backers and shareholders who have invested over $317 million dollars into lock-in and marketshare (Medium wears the same manacles).

That kind of investment really warps one’s perspective. ROI becomes the priority, with what’s best for the software and its users, a very distant second.