Barbarians at the Gate! Indies vs Big Publishing

The Fall of Rome is still debated. How could such an empire fall? Various theories are floated; taxes were too high, barbarians joined the army, borders became too porous, corruption and incompetence were rampant.

But I would argue that these were mitigating factors. Empires always fall for the same reason.

They stop adapting.

Adaptive Capacity is the technical term for an ecological or social system’s response to changing conditions in the environment.

A system that cannot adapt, self destructs.

Traditional publishing is just such an empire, built over half a millennium (if we go by the invention of the Gutenberg press) the industry has had a long run. Now, e book publishing and print-on-demand technology have changed the landscape. Within a short amount of time, the book market has transformed. Some of the new players are Amazon Kindle Direct Publishing, Kobo, Apple iBookstore, Barnes & Noble Nook Press and distributors like Smashwords and BookBaby.

Barbarians are at the gate, in the form of indie writers and upstart publishing imprints ( Amazon has thirteen). To combat the encroaching rebels, publishers have applied a series of strategies and exposed a growing divide in the the writing community.

Amazon vs Hachette

The Amazon/Hachette debate is not just a negotiation, it’s a skirmish between the new world and the old. The latest salvo comes in the form of a letter signed by a number of brand-name authors who support Hachette’s point-of-view. In response, a petition was circulated and signed by the indie writing community  supporting Amazon.Why did a simple business exchange create two opposing camps? Because the argument represents deeper, more treacherous currents between a crumbling empire and an evolving system.

Writer Douglas Preston circulated a letter signed by brand name authors such as Stephen King and John Grisham:

The Letter

An excerpt from the letter:

“As writers—some but not all published by Hachette—we feel strongly that no bookseller should block the sale of books or otherwise prevent or discourage customers from ordering or receiving the books they want. It is not right for Amazon to single out a group  of authors, who are not involved in the dispute, for selective retaliation.”

(The wording is ironic considering book stores have refused to stock Amazon titles. Even director Penny Marshall’s memoir “My Mother Was Nuts” was boycotted by Barnes and Noble, Wal-Mart and Target.) Any writer going outside the traditional system by publishing with Amazon is boycotted in the same way, no matter how many copies sold. So a clear line has been drawn; Amazon is an Indie too.

In response to Douglas Preston’s letter, Indie authors circulated a petition assembled by popular writer’s J.A. Konrath,  Hugh Howey and Barry Eisler:

The Petition

The wording of the petition put it succinctly:

“New York Publishing once controlled the book industry. They decided which stories you were allowed to read. They decided which authors were allowed to publish. They charged high prices while withholding less expensive formats. They paid authors as little as possible, usually between 2% and 12.5% of the list price of a book. Amazon, in contrast, trusts you to decide what to read, and they strive to keep the price you pay low. They allow all writers to publish on their platform, and they pay authors between 35% and 70% of the list price of the book.”

And though Amazon has offered Hachette writer’s 100% of e book profits while the negotiations rage on, Hachette has refused the offer and hostilities are still burning.

The power brokers of any entrenched system will predictably refuse to transform.  After all, they are the status quo and a new system is a threat. Publishing is an elaborate structure made up of authors, editors, proofreaders, executives, marketers, sales reps and agents, sustained by networks of critics, trade organizations, bestseller lists, library associations, trade publications, awards committees, distributors, brick & mortar stores and remainder’s dealers. Everyone of those players has a stake in the old system and evolving technology is slowly eroding that system. This has lead to an unprecedented loss of power and control among publishing companies and an encroachment of territory by the new system.

Data control

To understand the current battle, we need to go back in time.

Before Nielsen’s BookScan arrived in 2001, the only data publishers collected was the zip code of the brick & mortar store selling the book. Bookscan was initially greeted with skepticism, then suppressed. Publishers had already seen what happened when Nielsen’s Soundscan hit the music industry in 1991, irrevocably shifting the power base and heralding the rise of previously ignored music genres like rap and christian. The new data threatened publishing’s control of perceptions. By 2004, publishers were purchasing data at $100,000 per year and not letting anyone see it, including authors. Writers in the system report being stonewalled or receiving book sales reports that were six months old. This careful parsing of data served publishers well, allowing them to control the perceptions of writers and readers.

They were continuing an old tradition pioneered by the New York Times Best Seller List, which releases rankings, but not actual sales figures. These rankings are based on a mysterious process of unverifiable estimates and surveys of secret reporting bookstores. (Look at any touring author’s scheduled appearances and you can deduce which stores report). The list is manipulated by sending authors on tours to the reporting stores, where books are ordered in quantity for signings or by book laundering.

A good example of the instinct for manipulation within the publishing industry is the creation of the NYT Children’s Bestseller List. When JK Rowling’s book sales unexpectedly overwhelmed the list in 2000, taking all the top spots, they needed to take Rowling off the adult list because her sales undermined a system that brought in tremendous revenue from publishers who bought ads. So the NYT Children’s Bestseller List was created in order to accommodate Rowling’s swelling numbers. As the NYT editors at the Book Review put it at the time, “The change (in the NYT Bestseller List) is largely in response to the expected demand for the fourth in the Harry Potter series of children’s books.” 

So by the time barbarians showed up at the gate, publishing had built a bloated, convoluted system that relied on manipulation of data, brick & morter bookstores and an antiquated remainder’s system.

What could go wrong?

The Way Back Machine

The Internet Era arrived innocently enough. I’m not talking about Darpa and Cern and all the techno-history which reads like stereo instructions. I’m talking about the 90’s, when people started using email, a few blogs appeared and social media was still a glimmer in its mama’s eye. To give you an idea of how the big publisher’s felt about the Internet, a 1995 Newsweek headline read: The Internet? Bah! Hype Alert: Why Cyberspace Isn’t, and will never be, Nirvana.

So in 1993,  when an unanticipated competitor appeared in the form of an online bookstore called Amazon, most of the publishing industry thought, So what?

At first Amazon was just another outlet. Book superstores like Barnes & Noble were thriving and Borders was expanding its empire. To brick & mortar stores, the only threat Amazon presented was sales competition.

Publishers did not understand the technological frontier. The Internet was just another way to sell books. Amazon was an outlet, not a competitor. But Amazon’s CEO Jeff Bezos did understand the potential of the Internet. He was playing the long game and was willing to explore innovative programming, algorithms, data-mining and exciting new ways to track reader habits.

On the surface, the Internet was a communications curiosity, but revolution was in the air.  In 1993, the year Amazon appeared, the Internet only communicated 1% of the information flowing through two-way telecommunications networks.

By 2000 (when JK Rowling was being tossed into the kiddie pool) that communication figure grew to 51%.

By 2007  the Internet carried 97% of tele-communication information. 2007 was also the year Amazon introduced The Kindle, offering free e books in the public domain and cheap downloads of new books.

The Kindle sold out within five hours.

If there was a moment of awakening, this was it. The pressure on bookstores was immediate as you can see in the chart. Bookstore sales steadily decreased and bookstores began to close at an alarming rate.

A succession of strange events followed.

War

The first thing the big publishers did was go to war.

In order to fight the onslaught of cheap e books, five major publishers, Hachette, HarperCollins, Macmillan, Penguin and Simon & Schuster banded together with Apple who planning to launch the ipad (2010). They joined together to fight Amazon’s downward pressure on prices. To fight Amazon, these companies agreed to fix the prices of their books, violating anti trust laws in the process. (U.S. District Judge Denise Cote later said Apple “created a mechanism and environment that enabled them to act together in a matter of weeks to eliminate all retail price competition for their e-books.”

The courts ruled against the publishers with big fines.

Despite the attempt to control prices, Brick & Mortar stores continued to close and publishers became reluctantly dependent on Amazon for sales.  In a desperate bid to survive, Barnes & Noble introduced the Nook, which saved them from the fates of their titanic competitors, including B. Dalton, Crown and Borders, eventually forced to declare bankruptcy.

In late 2007, Amazon was also beta-testing a structure for writers to self publish, offering 70% royalties. In publishing circles, this was heresy. A  cold war started on the Internet. The old guard released a torrent of saber rattling blog posts, warning aspiring authors not to self publish. It’s dangerous! Reckless! they shouted, insulting Indie writers, calling them vanity press, substandard and illiterate. Like priests defending their temple, publishers, writers, agents, trade organizations and bookstores closed ranks. There was no way e book writers would cheat their way inside the hallowed gates of publishing. Publishers even fought for out-of-print back-lists previously left to rot. Bookstores refused to stock indie books or anything from Amazon’s Publishing Imprints.

The most unsavory aspect of this predictable reaction was the marked lack of concern for art. But publishing had jumped that shark a long time ago. (Anyone who’s ever walked into a Barnes & Noble and witnessed the geegaws and novelty books piled high on the bargain tables knows what I mean).

While big publishing tried to shore up its position, Amazon was innovating. They created their own bestseller lists and used their algorithms to match writers and readers. Mid-list writers who had been left to languish without any marketing help started publishing new novels online with Amazon. Indie stars started to rise. By 2010, Amazon allowed writers to see their Nielson sales figures. Agents and publishers reacted in a flurry of Twitter posts some in support, some against.

In 2010 Macmillan and Amazon’s negotiation over e book prices lead to delays in delivery, setting publishers on edge. The battle continues to morph into skirmishes like the current negotiation between Amazon and Hachette.

Simon & Schuster and HarperCollins contracts are also set to expire and they will have to negotiate with Amazon who supplies them with as much as 50% of their revenue. To see how the earnings landscape is changing, check out the Author Earnings Report.

Brave New World

So far, Big Publishing’s reaction to the brave new world of e books shows a lack of adaptive capacity. They have tried to force Amazon into capitulation with their old system rather than adapt to new market forces.

When technology is involved and those in power lack vision, a deadly form of myopic denial can develop. This denial of reality pervades the publishing industry at a time critical for its health. The more publishing refuses to look toward the future, the bigger the chance it will decline and collapse. Instead of Visigoths, Vandals and Huns, the publishing industry is defending itself from a strange and wily opponent, the future. The problem is publishing companies see this paradigm shift as an opponent rather than an opportunity.

There are good examples in history of what happens to corporations that lack adaptive capacity.

Eastman Kodak’s power seemed unshakable. Yet despite having a new and advantageous secret before anyone else in the industry, they did not take advantage of the fact. Instead, they chose decline. Kodak engineer Steve Sasson invented the first digital camera in 1975 and was met with a stunning level of resistance to his invention. In the NY Times he describes the corporation’s reaction “But it was filmless photography, so management’s reaction was, ‘that’s cute—but don’t tell anyone about it.”  Kodak, who had built their fortune on film, chose to license the digital technology to other companies, actually supplying competitors with the weapon that took them down, the digital camera.

Why would a company whose own engineer comes to them with a piece of technology that will revolutionize the industry not see what is in their hands? Why would they tell the inventor to keep quiet? Why would they think they can suppress an invention?

Many industries have had to drastically reorganize when facing similar circumstances. The film industry was nearly toppled by talking pictures, then by television. The music industry was thrown into disarray by point of sale scanning reporting real sales figures, MP3 technology, itunes, ipods and CD Baby. In the 19th century, the Luddite movement was a violent reaction to machines where workers ran around destroying them with sledgehammers. This has all happened before. When fear rules, businesses circle the wagons and try to quash new technology and sometimes, new voices.

Freedom

Indie publishing has freed writers to work on their craft by publishing and interacting with readers, rather than languishing as they seek approval. The writer John Kennedy Toole is one of the more damning statistics of the old publishing system. His brilliant novel Confederacy of Dunces went through several revisions with editor Robert Gottlieb at Simon and Shuster who felt the comic novel was “Pointless.” After a series of rejections from Simon and Schuster and Hodding Carter Jr., Toole fell into a black depression and committed suicide. Years later when his mother brought the manuscript of Dunces to the attention of novelist Walker Percy the book was finally printed and Toole posthumously awarded the Pulitzer Prize for Fiction. This tragic story is the primary reason why Amazon’s treatment of writers has been a revelation. Voices the publishing industry’s system would have unwittingly squelched are being heard. Writers need to write for readers, not try to please executives.

If e books had been around for John Kennedy Toole, he might have survived.

Amazon is adaptive, they have created a new delivery system and a new content system. They are developing publishing imprints, they are using data to find partners, they have found ways to encourage writers to hone their craft.

So far, Publishing is not adaptive. They’ve stopped addressing customer needs, become product oriented not customer oriented. They’ve forgotten why they got into the business in the first place. Publishing should respond to the needs of readers by finding new voices and communicating the human spirit.

Empires fall, and strangely, people don’t appear to learn from history.