On the Slow Adoption of EPUB 3
The slow adoption of EPUB 3 was a major thread to come out of the Publishing Summit at W3C’s TPAC, even though it was recommended in October 2011. I have been trying to puzzle out why it hasn’t been more widely adopted and am still unsure, to be perfectly honest.
But let’s back up a wee bit. What are the major differences between EPUB 2 and EPUB 3?
- One of the primary changes is that the spec allows for rich media: audio, video, media overlays and read-along. People who had been dying to embed a/v content rejoiced, until they found out that playback was supported in only a few reading systems.
- EPUB 3 is also marked by adoption of HTML 5 and CSS 2.1, where EPUB 2 relied on XHTML 1.1 and a subset of CSS 2.
- Support for MathML and SVG in the spine are a foundational part of EPUB 3.
- EPUB 3 supports javascript baked right into ebooks.
- For people down in the weeds creating EPUB, it means no longer needing to wrestle with the NCX document for navigational purposes. The navigation comes from an HTML document, instead of the NCX – although the latter can still be included for backwards compatibility.
- And finally, navigation. EPUB 3 specifies the possibility for more thorough and multiple means of navigation via traditional TOC, a print-corollary page list, and via landmarks and
epub:typesemantics.
The most important benefit of EPUB 3? Enhanced accessibility. EPUB 3 allows for deeper meaning to be applied to the various elements of an ebook via features like epub:type semantics (which aligns with ARIA roles) and HTML 5 tags. EPUB 3 content is more machine-readable and therefore more legible to assistive technologies.
Barriers? What Barriers?
So what is it then? Why is 70% of what is currently posted for sale on major (and minor) retailers EPUB 2 instead of the more current, more agile, more accessible EPUB 3? Here are some of the reasons I’ve heard while poking around on Twitter:
- There is not enough reading system support for EPUB 3.
- Many tools create EPUB 2 by default.
- The benefits of EPUB 3 aren’t clear and so developers see no reason to change.
- EPUB 3 looks unnecessarily complex.
- Lack of client demand for EPUB 3.
Let’s address these concerns one by one.
There is not enough reading system support for EPUB 3.
All major and minor retailer, distributors, and aggregators support EPUB 3. Please correct me if I am wrong. But I think this is an education issues. I wonder how much people don’t realize that the situation has changed in the last five years. Support was very slow in coming, true, but I don’t think that’s true at all in 2017. And, as EPUB 3 allows for an NCX so that your EPUB is backward compatible when a reader is using an EPUB 2-only reading system. So then a basic EPUB 3 will be fully supported. Go out into the world and do some testing – I promise you will be happily surprised.
Many tools create EPUB 2 by default.
The most broadly used tool in a space like trade publishing is Adobe InDesign, which exports perfectly reasonable EPUB 3. Sigil works with EPUB 3 without difficulty (and, in fact, has a great EPUB 2 to 3 plugin). If you are making EPUBs from Word, it is easy enough to opt for EPUB 3 with most converters. In fact, Kindle Previewer plays well with EPUB 3.
Ebooks are more accessible by default as EPUB 3. They will be easier to navigate and so easier to read.
The benefits of EPUB 3 aren’t clear and so developers see no reason to change.
EPUB 3 is a better, more agile format. If you are making fixed-layout content, you need EPUB 3. If you want bells and whistles, you need EPUB 3. But if those particular features don’t concern you, you might be interested in the fact that your ebooks will be more accessible by default as EPUB 3. They will be easier to navigate and so easier to read.
EPUB 3 looks unnecessarily complex.
Ebooks made with an EPUB 3 wrapper are slightly more complex and will require more attention, particularly in the area of semantics. But like any bad habit, twenty days of practice means that habit is replaced with a new, healthier one. In fact, using the more specific HTML5 tags afforded by EPUB 3 – article, section, figure, figcaption – will provide the semantics and so make a developers job easier as you don’t have to think up endless class attributes for all those <div>s.
Lack of client demand for EPUB 3.
Lack of client demand for EPUB 3 is a funny one. Publishers are, ahem, a notoriously conservative, slow-to-change group. But they move when pushed, as I know from having done it many times. Bamboozle them with big words, if you have to! I suspect that a big chunk of these EPUB 2 files freshly uploaded into the marketplace are self-published. And self-published authors are not likely to have the technical knowledge to ask for EPUB 3, to be perfectly frank. Waiting for a self-published writer to push developers to be more experimental might not work out for anyone.
Publishers are, ahem, a notoriously conservative, slow-to-change group. Bamboozle them with big words, if you have to!
So What Are You Waiting For?
Do it, jump in! Don’t be scared. The reading systems support you. The distribution aggregators are on your side. Your fellow ebook developers will even hold your hand through the switch over, if you like (see #eprdctn).
It seems to me that reluctance to move to EPUB 3 is driven in part by lack of education about the issue. Do a little testing, poke around in the format. Ask fellow developers about their experience. Test an EPUB 2 with voiceover and compare the results with a well-built EPUB 3 file. The difference in that experience might just be the tipping point.
What’s holding you back?
One note about InDesign export: EPUB 2.01 is the default. When you go to File > Export, you’ll see the General panel. Just pull down the menu next to Version and choose EPUB3. InDesign will create the toc.ncx for you along with the new standard navigation file, so the file will be compatible with ancient devices.
Excellent, thanks for this article.
Let’s not forget the useful (but hard to grok) chart of EPUB3 feature support by ereaders by the BISG/IDPF, which appears to still be maintained: http://epubtest.org/testsuite/epub3/
Thanks for the article. Part of the problem is the lack of Kindle support in the MacOS and iOS Kindle apps. iBooks supposedly works so well, but in practice, I see more difficulty in using it to actually read. They seem to be dying a slow death—though their stats are still respectable. For most authors I know, iBooks, Kobo, and Nook provide virtually negligible portions of sales. The big need remains—an ereader and company which can take on Amazon in a meaningful way. Amazon is the limiting factor. If InDesign could only provide an excellent Kindle export that worked…
But the real problem IMHO appears when you start talking with readers and discover that no one cares about the fancy abilities. Audio, video, rich media and all the rest are not sought after by any large reader niche. They change the book into something else which is neither fish nor fowl. We see yet another “you’ve done a wonderful job of developing something about which I care very little” scenario.
David, I agree that many of the EPUB3 bells and whistles aren’t needed in most published material. But authors/editors/publishers should understand that EPUB3 navigation is accessible, which only increases market share. And it’s not hard to include!
Libraries and schools have a legal mandate to make all materials accessible. Enforcement has already been felt via lawsuits against libraries and universities. So, more and more they will be NOT acquiring EPUB2 books.
EPUB3 books with accessible navigation and semantics are MORE book, not less, not some strange hybrid. And audio, video, etc. do have a place in books, but in the right circumstance.
This is something that publishers should care a great deal about.
My take as a writer, editor and publisher rolled into one.
1. “People who had been dying to embed a/v content rejoiced…” And most of us yawned. Such content is expensive to produce, license and embed, particularly when what you’re creating is a book for which A/V is often a distraction. I can’t think of a single one of the 30+ books I have published that would have benefited from A/V. And I have never seen any indication that readers care either. When they want to read, they want to read.
2. “All major and minor retailer, distributors, and aggregators support EPUB 3.” Not even close. Amazon owns at least 60% of the ebook market, and it has its own format. It does seem to know how to covert my EPUB 3 to its format, but there’s little reason for publishers to diddle with EPUB 3 features that won’t make their way onto Kindles. Like it or not, the Kindle rules.
3. “Lack of client demand for EPUB 3 is a funny one.” Not funny at all. There are HTML features that were lacking in EPUB 2 that I’ve love to see in EPUB 3, handy features such as accordion text. The fact that EPUB 3 did not add them is why it isn’t meeting my needs. I still publish in EPUB 3, but that’s because it’s an easy choice with InDesign, not because I am enamored with it.
What is my basic gripe with EPUB in all its incarnations? None give me the power to make a digital book look as good as I can make a printed one look. And no, the reflowable bit is not a valid excuse. It would not be hard to give reflowable enough AI to follow the usual layout rules that make a page look good. Nor would it be impossible to allow those who do book layout to specify how a graphic is handled in all screen sizes. Framemaker was intelligently handling graphics in reflowable text thirty years ago. Add a page of additional text on page 37, and a graphic on page 337 would intelligently move to where it belonged on page 338. Intelligently directed layout isn’t rocket science.
As I often point out, the first movable-type printed book, the Gutenberg Bible, looked as beautiful as the illustrated manuscripts of that day. Note the contrast. What do most of today’s ebooks look like almost twenty years in? Like ebooks looked on those long-ago Palm Pilots. Except for fixed layout EPUB, they don’t look remotely as attractive as a well-done print book. You can’t even make them look good. The tools aren’t there.
My advice? Make EPUB 4 genuinely useful for creating attractive ebooks and those who’re stuck at EPUB 2 will skip over EPUB 3.
> EPUB 3 supports javascript baked right into ebooks.
That’s a reason not to upgrade to EPUB 3 right there.
(Goes off to write a script to see how many of his ebooks contain script elements)
> writes a script to look for script elements.
… and the answer is : 4 books.
and two of them are broken, and one of them the JS is for debugging.
Only one book: “The Swift Language Reference 1.0” has actual executable JS in it.
> That’s a reason not to upgrade to EPUB 3 right there.
Some retailers are allowing EPUB 2 files to have JavaScript too.
Actually, there’s nothing preventing scripting in EPUB 2 files except ePubCheck (and the app itself disabling scripts before or during runtime for EPUB2/EPUB3 files).
FWIW, I even dealt with Apple’s Security Team next year because I reported a small security flaw which had gone unnoticed for years… the TL;DR is that it’s far more secure than email for instance.
From my point of view, as a software developer, I think that one of the biggest problem is lack of libraries with support to ePub3.
For instance, for Java, there’s a popular library called epublib (https://github.com/psiegman/epublib) and it only supports Epub 2 (see https://groups.google.com/forum/#!topic/epublib/A9uEgaQ5vzo)
Anyone know a reliable alternative to this library with support to ePub3?
Sounds awful that it includes support for Javascript. This might cause ebooks to get massively bloated like the web has.
I am a newcomer to epub3, have been dabbling with it for a few months, trying to learn the format since I am trying to make a comic book with some animated panels. Unlike others above, I find the option to have video, sound and interactivity with Javascript a great option for those who want it. The format has the potential of being a great multimedia platform in the future.
Unfortunately I have seen a few big problems. The main ones are (and apologies because this is going to get technical):
1- The format not adopting full CSS3 was a mistake. CSS3 introduces new properties that eases a lot the burden of making complex layouts, like centering a DIV on a reflowable page.
We can’t do this:
.divToBeCentered{
position:absolute;
width:35%;
height:35%;
left:50%;
top:50%;
transform: translate(-50%, -50%) }
since CSS2.1 does not include the transform property. This is the easiest way I know to center a div on a reflowable page and unlike other ways I have tried works on all the major browsers.
2- To my experience, most readers do not support well the epub3 format. Only a subset of it. I cannot write in CSS the property of an object as absolute without most readers I have tried going haywire, for example.
The International Digital Publishing Forum should have a more heavyhanded posture when dealing with e-readers developers. If they announce they support the epub3 format, they must support the ENTIRE format, not just a subset of it. If they don’t do it, their license to claim to be epub3 compliant should be revoked. If you follow the format rules, and a page renders well on the major browsers, it should render on epub3 readers as well.
I would like to be more positive, but these two problems make my experience with the format disheartening. I would like to know other people experiences.
Eduardo, you make good points. It is in the hands of the reading systems to implement EPUB3.
One thing, though, the IDPF is no longer in charge. The W3C and IDPF came together earlier this year (I think it was) and now EPUB development is being done by working groups within the W3C.
Maybe the W3C will have the muscle to force reading systems to include more features. But, we’ll still have lots of reading systems from many vendors with lots of variety, including older versions of devices in public settings like libraries. So, it’ll be tough to get full compliance for every reader.
You are right, Kevin. The IDPF and W3C merger was earlier this year in February. Somehow, I had the idea it was yet to happen.
I am a bit skeptic about the ability of the W3C to enforce standards. Their policy of publishing standards without releasing a (preferably open-source) working proof of concept is an invitation to bad standard implementations (Internet Explorer 6 anyone?). If they released a working implementation it would send a clear message saying “you must make an implementation at least as good as ours”.
At least for IDPF was wiser and released Readium working as an implementation. Unfortunately, from my experience it has some serious bugs when it comes to render more complex layouts.
As for my project, I will not wait for the W3C. As I see things, the best course o action is to clean a little bit my prototype and then send it to Readium developers in a Bug Report, so they can sort things out. If they fix the issues, at least I can put on the website: compatible only with Readium v. XX or later based readers .
[…] Pourquoi l’EPUB3 semble tarder à s’imposer ? EPUBSecrets en a rédigé en ce mois de décembre 2017 une note intitulée On the Slow Adoption of EPUB 3. […]
Well, I can reply to that.
(disclaimer: I am the Readium CSS project lead so I guess I can at least discuss the CSS parts. It’s worth noting we’re currently re-starting from scratch so if you have needs or requirements, it’s better do file them in the Readium 2 repo so that we can take them into account ASAP: https://github.com/readium/readium-2.)
First and foremost, EPUB 3.1 is using W3C snapshots (see https://www.w3.org/TR/css-2017/), which sets a baseline for specs we should support – and we will support EPUB 3.1, we’ll even support specs which are not yet in the snapshot.
Does this mean everything CSS3 will work in reading apps? I’m afraid not. Please allow me to clarify.
Building a rendering engine from scratch would cost dozens of millions of dollars. As a consequence, reading apps willing to support EPUB 3, in their vast majority, are going to just use those of web browsers. As a matter of fact, that’s even a use case for desktop and mobile platforms, since they’re making web views available (i.e. a web browser without the UI).
In practice it means that 1) the reading app will add the back-end to process EPUB files, which are not natively supported by web browsers and 2) build a layer on top of those rendering engines (UI, features like annotations, user settings, toc, user settings, etc.). Which brings us to pagination…
The only implemented spec dealing with fragmentation right now is CSS multicol so reading apps will use that for pagination. For the record, Apple even extended CSS multicol a little bit to create a pagination API (Application Programming Interface) on iOS.
Truth is CSS multicol is a super simplistic model, with notorious limitations, WTF bugs, etc. So absolute positioning might not work as expected for instance.
We would like to solve those issues, really, but it can only be fixed at the rendering engine level like, say, page-break. To be super clear about this, it can only be really fixed by Apple (Webkit), Google (Blink), Mozilla (Quantum) and Microsoft (EdgeHTML), and we’re completely dependent there.
As for EPUB 3, there’s stuff in the spec we simply can’t implement because, in real life, it won’t work – I assume you would not be happy if your files would take 10+ seconds to display. Indeed some parts might simply impose too much of a burden. And there are parts of the spec on which implementers can’t agree either because it’s too vague or the articulation has a lot of conflicts, which creates interoperability issues.
The web’s solution to this overarching issue is WHATWG a.k.a. web browsers’ vendors discussing interoperability and practical implementations, and making it clear they can’t support some specs or will support them in another fashion. Since we don’t have such an “organization” for EPUB (yet), everyone suffers.
Please be assured that we’re having an insane lot more issues at the Reading System level than EPUB authors, and we would like to solve them (see https://github.com/rachelandrew/multicol-wip/issues, in which 50% of issues are my own, and https://github.com/w3c/publ-cg/issues, in which I’ve been one of the few active contributors in the last 3 months), but if we want to fix things in practice, 1) at least the major Reading Systems should discuss to unfuck the current situation and 2) we’ll have to work with browser vendors to make it better.
Now, I’m an EPUB author as well – and, FWIW, I also created an EPUB 3 framework. You might want to use flexbox there, as demoed in http://epubsecrets.com/easy-css-wins.php and listed in https://friendsofepub.github.io/eBookTricks/.
Hi Jay
Thank you for for your very informative post, I will follow your recommendations and feel more optimistic now. 🙂
I apologize if my previous posts sounded whiny towards Readium developers, that’s not what I intended it to be.
Somehow I missed EPUB 3.1 was using W3C snapshots. I read the announcement but since it was a .1 update I assumed it did not take a more “dramatic change” from CSS 2.1 to 3.0. My bad.
The Google searches I did also gave CSS 2.1 as the base for epub3. Mental note: “be careful with old content on the internet”. 🙂
I had realized before the best course of action to make a epub reader these days would be taking a browser rendering engine and adding an epub-specific layer to it. The main ones do a decent job, so why reinvent the wheel?
I also realize trying to make a comic book using unorthodox methods like doing the layout using HTML+CSS is asking for trouble 🙂 but since I want the content to be reflowable, I don’t see another way to do it without using media queries, which I want to avoid since it means it will be multiply the work when it comes to writing CSS rules.
To tell the truth, most ebook readers I tried my prototype on were Readium-based, but none of them rendered it quite the same. I suspect they took a snapshot during Readium development and rarely bother to keep it updated afterwards. Oh well… 🙁
I share with you the same frustration towards web standards. Everything seems designed by committee and that too many cooks are working on it. I understand there are lots of corporations/egos to appease. In the end, like you said, there are WTF bugs, the spec is vague and we have to use hacks to get the desired effect. The W3C needs to return to the KISS principle.
I will check you epub3 framework. Hopefully it will help me on my journey. 🙂
As soon as I get Readium 2 working on my computer I will start to test it and tell you and your team if there is something wrong.
Once again, many thanks for you post! 🙂
[…] does anyone else want change? Laura Brady wrote an article on the slow adoption of EPUB 3 just last month; the very same EPUB 3 which became an official recommendation in October of 2011. […]
is epub3 adopted now in 2018 or it still struggle?