Last week, the W3C and the IDPF—the organizations responsible for the HTML standard and the EPUB ebook standard respectively—finalized their plan to merge. The plan has not been without controversy. Proponents have said that it will give the IDPF the reach and power of a larger organization to push for its improvements in the ebook standard, whereas opponents have been concerned that it will lose its identity, and publishing-related organizations will lose their voice, as the IDPF is subsumed into the larger W3C.
Last month, Digital Book World carried dueling editorials from people against and for the merger. The opponents, including execs from Overdrive, Impelsys, and Open Road Integrated Media, would prefer to see the IDPF revitalized on its own, and have a ten-point plan they want to implement to make that possible.
On the other hand, long-time IDPF member Bill McCoy thinks that this merger could move EPUB to the mainstream, rather than letting it peter out like the WAP mobile web implementation did when everyone else moved to HTML5. He also feels that incorporating EPUB directly into ongoing web development could allow ebooks to move beyond “ebooks that are skeuomorphic digitized equivalents of print editions.” The bigger organization will have more people doing the same work and a louder voice to be heard, and McCoy adds that using the W3C’s royalty-free licensing model will ensure that EPUB remains free and open for all to use.
In a CNet editorial, Stephen Shankland anticipates that this could potentially bring all sorts of “smart” features to digital textbooks, such as pop-up tables, embedded video and 3D models, and so forth. Such textbooks could be updated automatically without requiring the repurchase of a new edition. However, The Digital Reader’s Nate Hoffelder points out that iBooks has already supported this sort of thing since 2012, as have a number of failed textbook startups. It’s not exactly new, but neither is it in very wide use.
But interactive ebooks? pop-ups? embedded videos?
We have all that, and ebook developers can use it when they want. The reason they don’t is that frequently the market doesn’t want it.
It is true that Field of Dreams had it wrong: just because you build it doesn’t mean anyone will come. But I foresee another problem with all this pie-in-the-sky talk of a broader EPUB standard for everybody.
When I was talking to David Rothman about possible angles I might use in this piece, he suggested that Amazon’s recent changes to its Kindle ebook format—making it more difficult to convert even DRM-free titles into other formats—might show the need for a more powerful standard-setting organization.
But here’s the problem with that. What do you call a standard that almost nobody actually uses? Yep, that’s right…you still call it a standard. And EPUB is exactly that kind of “standard” that effectively isn’t one.
Perhaps it’s not entirely correct to say that “almost nobody actually uses” EPUB. Because, really, just about every ebook vendor who isn’t Amazon uses EPUB. The problem is that the vast majority of ebook customers use Amazon, whose format isn’t compatible with EPUB. (Or, to be more precise, whose format can’t be (legally) made compatible when DRM is involved.) So, as far as all those readers are concerned, Amazon’s non-standard standard is their standard standard.
We’ve seen multiple ebook businesses crash and burn (most recently, Shelfie) because they simply couldn’t get enough customers—their would-be customers were already strolling through Amazon’s garden. And while it might not be a completely walled garden—customers can still sideload books from elsewhere if they choose—the way it works out in actual practice, it might as well be.
To the average person, sideloading stuff is hard, or at least too inconvenient to want to mess with. Even if other ebook stores released their ebooks in Kindle-compatible DRM-free Mobipocket format, most Amazon customers couldn’t be bothered to step outside their tap-a-button-to-get-a-book comfort zone. (That’s why Baen neutered its own ebook store’s subscription bundles in order to get onto Amazon—that’s where all the customers are.)
So, even if the newly merged W3C/IDPF is the most powerful method ever for promoting EPUB as an open standard, what good is that going to do? Amazon already has all the customers, and it’s not interested in switching things up. EPUB can be as “standard” as it wants to, and the W3C can use as big a megaphone as it wants to tell everyone how awesome it is—but that’s not going to pry one customer loose from Amazon when that’s where all his or her books are. Nor is it going to force Amazon to add EPUB capability to the Kindle if Amazon doesn’t think that’s in its best interests. It would probably take some kind of antitrust action to change that, and as long as the current “consumer welfare” paradigm of antitrust enforcement holds sway, I don’t see that happening.
I hope something about this surprises me, and the newly-merged W3C means some kind of sweeping change will come about that could revitalize the e-publishing industry and change the balance of power. But I suspect it’s going to go right on being business as usual.
Correction/Update: Firebrand Technology reversed its opposition to the merger, as Chair Fran Toolan mentioned at a January 18 meeting in New York, and in fact has provided a royalty-free grant in support of the transfer of EPUB IP to W3C. We’ve removed Firebrand from the list of merger foes in an earlier version of the post. Others may also have changed their minds.
teleread.org 
@Chris: Good commentary. May I respectfully disagree on a key point, however? America is not the world. I highly doubt that Amazon controls the majority of the global ebook market. ePub is probably faring better there. What’s more, the Kindle formats are almost surely not beating ePub in the public domain world. Luckily, when we talk books, we can look beyond “markets.” So, yes, ePub and Publishing@W3C do count.
That said, from an antitrust perspective here in the U.S., it would be wise for Jeff Bezos to become less proprietary, not more. In the ebook area, he’s setting himself up for The Orange One’s anti-trusters to turn the Trump-Bezos feud into reality if they discover what’s going on.
Thanks,
David
LikeLike
America’s book marketplace is far, far larger than any other country’s, but it’s only about 28% of the global book market.
For ebooks, however, it’s a larger percentage — for the moment.
LikeLike
Thanks, Marion. I’d love to see some decent stats, but whatever the case, yes, we need to look beyond “the moment.” Traditionally around half the readers of TeleRead have been from outside the U.S. And remember, our main readership is English speaking. Goodness knows what’s happening in China. David
LikeLike
David,
For global industry size, check I don’t know how accurate it is, but the total of US$106 billion is somewhere in the right area. And the US publishing industry consistently runs at least US$28 billion.
That’s just one reason that ebooks as a viable commercial format launched here. BTW, the English language publishing industry is also far, far larger than the next biggest language, because the US, Canada, the UK, Australia and New Zealand combine to be a very large bloc.
LikeLike
Continued appreciations, Marion. Did a quick Google. Here’s a PW article on the digital publishing biz in China. And here’s the lowdown elsewhere about India. Bottom line: Even if the U.S. offers most of the action right now, yes, things will change—hence the opportunities for ePub, in overseas markets where Amazon is not as strong as here.
LikeLike
And given that Amazon is basically everywhere English is the language people buy ebooks in, its influence will be disproportionately large in that regard.
LikeLike
I only can agree. One example: Russia. Dominant e-book store is Litres.ru they offer any format customer wants (including EPUB) but main format is FB2 (sometimes they are forced by publishers to offer DRM-only solution – in this case they use Adobe DRM on EPUB or PDF, their ‘library’ solution just uses their own app for iOS/Android/WinPhone). Customer either download file or use their mobile app to read on ios/android/winphone/supported eink device.
If FB2 can’t be used because it doesn’t support necessary features like fixed layout, PDF is used instead.
Almost all eInk readers sold in Russia support FB2 and EPUB, some even support unDRMed EPUB, some android-based eInk readers have Litres’s app preinstalled.
Other stores usually sell either PDF or FB2+others (Conversion FROM FB2 to other formats is standartized and yes, this mean many fancy features are not available)
LikeLike
Chris, even though Amazon doesn’t use epubs, that doesn’t mean that publishers disregard it.
My guess (based on my own experience as a publisher who produces ebooks for Amazon and other stores) is that the most common workflow is to produce an epub file which can be cleanly converted into a kindle file and used for other bookstores.
The Kindle Previewer (produced by Amazon) explicitly performs conversions between epub files and .mobi files. There really isn’t an alternate path for publishers to create source files and convert them cleanly into .mobi files. (Possibly through Calibre you could go from .docx to .mobi ? I don’t regard this is as a clean solution).
Before 2013 or so I would say that .mobi flaunted the epub standard. Amazon’s implementation of .mobi was downright bizarre — and weighed down by legacy issues. With KF8 and a very user-friendly Kindle previewer tool, the clash between the two standards had effectively diminished to next to nothing. Sure, Kindle didn’t implement several key aspects of epub3, but they did embedded fonts, some SVG, media queries; good enough for me! It’s conceivable that they could accept only epub file at some later date. The thing is, the readers wouldn’t ever need to know!
Whether or not the merger occurred, there would always be a disconnect between what the standard is and what reading systems are supporting. The W3 ensures that the browser people are on the same table as the reading systems people. My fear is that W3 will implement features more quickly than reading systems can support them. Another fear is that W3 discussions are more inclined to assume persistent web access. For the epub3 standard, offline access is very important; with w3, I don’t think they view this as a high priority.
Another thing is that Google exerts a lot of influence over W3. They may push for things convenient to their business interests or conversely ignore certain needs that publishers have. On the other hand, this might result in closer alignment of web standards with ebook standards. That is good, because it gives publishers access to more general purpose tools for debugging and testing.
Future improvements to epub standards might relate to creating interoperable bookshelf and catalog systems. That has more to do with infrastructure rather than standards for individual books. What can be embedded in an ebook? How do ebook systems connect with browsers and other apps? I think W3 is better able to hammer out these kinds of issues.
PS, great word — skeuomorphic.
LikeLike
Chris,
According to my understanding, this has little or nothing to do with Amazon. Instead it has to do with two things.
One is the Portable Web Publication (PWP) standard (https://www.w3.org/TR/pwp/) that the W3C is already working on. At the risk of gross oversimplification, PWP is intended to be an HTML-based format for self-contained publications that can be read offline, thereby obviously overlapping with EPUB. There’s a small number of people who are involved with both PWP and EPUB, and they think it’s a good idea for one standard to emerge for this rather than two separate ones.
I had been skeptical about the future prospects of PWP. Good people are behind it, but the momentum behind any new working group within a large standards body is always something to be wondered about. But then, at the recent DBW conference, a “highly placed publishing executive” gave me a raison d’etre for PWP that makes total sense: technical documentation from huge tech companies that are in the drivers seats at W3C. They want open standards for their voluminous tech docs and generally to make them as easy and cheap to produce and distribute as possible. Ah-hah – now I get it.
The other is a lot simpler: money. I think the IDPF leadership just got sick and tired of having to beg and plead and scratch and scrape for funding for everything they’ve wanted to accomplish (having been a recipient of such funding) and are looking forward to being part of an organization where more resources are available. Notice how the IDPF has been trying to raise interest in EPUB for corporate documents instead of commercial publications — which is also reflected in the agenda of the EPUB Summit next month in Brussels (https://www.edrlab.org/epub-summit-2017-program/). There’s a well-trodden path of technologies for commercial digital media toward the more lucrative world of corporate enterprise applications (DAM/content management springs to mind).
The tradeoff is the actual publishing industry’s diminished prospects for influence over EPUB and related standards. I know of only two publishing companies that are W3C members (Hachette and Wiley). Membership in W3C carries a much higher price tag than IDPF membership. The organizations have taken some steps to define a lower-cost W3C membership status for incoming IDPF members, but it’s unclear (to me, at least) how this will play out in the long term, whether actual publishing companies will remain involved and what control they will have.
LikeLike
@Bill: Appreciated your informative comments.
I myself would still want to consider the Amazon factor even if that wasn’t an announced reason for the merger. The more attractive ePub or a successor can be, the better for readers and publishers alike as an alternative to Amazon formats. The merger will give ePub folks a lot more resources. Among other things, W3C needs to spend serious money to educate nontechies on the considerable long-term risks of proprietary formats.
Also, let’s hope that the W3C folks will understand the usefulness of current self-contained books and of compatibility, and that booklovers, content-providers and developers will suffer minimal disruption. It’s a risk I’m willing to face.
I myself would hope that ePub products such as Moon+ Reader Pro could deal with PWP if that emerges as a unified standard for publications whether the main readership is brower- or ereader/app oriented. If not, ePub needs to stand apart. I have yet to find a browser-based reader with all the amenities and smooth operation of apps like Moon.
The other thing is that I hope there’ll be lots and lots of open-content creation tools for the new ePub, PWP, whatever. With W3C’s greater resources, I see more of a chance of that happening.
Thanks,
David
P.S. In an era of networked books, I can see PWP opening up some interesting possibilities for browser-oriented users beyond those that ePub offers. But again, let’s not diss self-contained books or forget about the need for a smooth transition.
LikeLike
Quote: My guess (based on my own experience as a publisher who produces ebooks for Amazon and other stores) is that the most common workflow is to produce an epub file which can be cleanly converted into a kindle file and used for other bookstores.
—–
My experience too. I use InDesign to create an epub and send that to Amazon for conversion. Except that the images don’t look as good in a Kindle format as ePub in the iBookstore, that seems to work well enough.
What ebook publishing needs is for ePub to acquire HTML-like abilities so it can handle more useful page structures rather than just novels and simple biographies. I’ve got print books that I can’t bring to digital because they’d look awful, given the inadequacies of ePub. Even adding an HTML basic such as accordian text would be helpful, as would showing a bit of intelligence with graphics, widows and orphans.
Whether this merger will accomplish that or not, I don’t know. I’ll just wait and see.
—–
The real problem with ebooks is that Amazon doesn’t care for anything outside itself—and it cares not for the overall health of ebooks. My hunch is that if we could see Jeff Bezos’s grade school report cards, they’d be filled with “Does not play well with others.”
My grade school cards typically commented on what was once called poor penmanship. That hasn’t improved with age. Except when I force myself to do block writing, my handwriting has gotten so bad, I can’t read much of it myself. Something similar seems to be true of Bezos and playing with others.
LikeLike
The reverberations of this merger will indeed be fascinating to watch. I expect that there will be many surprising developments along the way.
The W3C itself has gone through a rough period where they lost control of the HTML standard to web browser vendors intent upon monopolizing the web for financial gain. Wrestling that control back was a messy affair but they did it with a lot of help from web developers and end users.
So who will take on the ePub equivalent of the Web Standards Project (http://www.webstandards.org)? Where will their support come from? Who will contend with them?
LikeLike
Why do we need ebook standards? There are standards for print books, legibility and price being at the top of the list, but Amazon fulfills those in ebooks for now (though much work to be done on legibility, accessibility, and more). But my point is why should ebook standards be determined by software. Why should software take the center stage when what we really need is people able to read and write? That’s what we should be focusing on. All this gibberish about epub is getting really old.
LikeLike
@Carl: A major reason for ebook standards is so that software will not have to take the center stage! Proprietary standards create all kinds of confusion and extra work for users. As for “people able to read and write,” libraryendowment.org might be of interest.
LikeLike
I appreciated Bill Rosenblatt’s mention of Portable Web Publication (PWP). I do hope that the PWP vision is realized in this merger between W3C and IDPF.
One of the things I imagine coming out of this is discovering a web based resource and being able to tell my computer to save it to PWP/ePub or whatever it might be called then. They’d be a bit of web spider to this with options to configure the nature and extent of the crawl, including how to handle images, video, etc. It would go far beyond Pocket et. al.
LikeLike