Institutions, Confidence, and the News Crisis

December 2, 2011

Dean Starkman has written a lengthy piece in the Columbia Journalism Review, assessing the writings of a group of us he calls the “Future of News” movement. That essay, Confidence Game, focusses principally on Jay Rosen and me, both of NYU’s Carter Institute, and Jeff Jarvis of CUNY, though noting some similarity of vision with Emily Bell of Columbia, Dan Gillmor of Arizona, and John Paton, publisher of the Journal-Register Company. (Unmentioned fellow travelers include, mutatis mutandis, Steve Yelvington, Chris Anderson, Amanda Michel, Steve Buttry, Jonathan Stray, and Alan Mutter.)

Starkman doesn’t just criticize us (though he does that, at length.) He also puts forward a Burkean defense of institutional tradition as a store of embedded wisdom, arguing for the continued relevance of existing news organizations, especially newspapers, in something very close to their current form.

He jokingly calls his vision the “Neo-Institutional Hub-and-Spoke Model.” His description of what’s needed, though — “rebuilding or shoring up institutions” — makes it clear he doesn’t need the “Neo-” bit. He is talking about somehow saving the familiar institutions, not inventing new ones, a strategy that has long passed for Plan A in the conversation about what the internet changes about the news business.

He’s not even wild about the familiar institutions altering themselves too radically to accomodate those changes. Paton, who is trying to save local news reporting, is derided for being a “FON practitioner” (shades of the 5th column), and The Guardian, a storied paper since back in the day, gets ten with the cane for going ‘digital first‘. When echt newspaper guys like Paton of the Journal-Register and Alan Rusbridger of The Guardian are seen as heretics, you know we’re talking about that old-time religion.

Rosen and Jarvis and Bell and I disagree plenty, but one belief we have in common is that the way newspapers used to be organized and funded is a bad fit for the current environment, and getting worse. More than any individual sentiment we may have expressed, our public loss of faith in the institutional logic of Plan A seems to be what has most aroused Starkman’s ire.

* * *

Institutions reduce the choices available to their members. (This is Ronald Coase’s famous argument about transaction costs.) This reduction allows better focus on the remaining choices they face.

The editors meet every afternoon to discuss the front page. They have to decide whether to put the Mayor’s gaffe there or in Metro, whether to run the picture of the accused murderer or the kids running in the fountain, whether to put the Biker Grandma story above or below the fold. Here are some choices they don’t have to make at that meeting: Whether to have headlines. Whether to be a tabloid or a broadsheet. Whether to replace the entire front page with a single ad. Whether to drop the whole news-coverage thing and start selling ice cream.

Every such meeting, in other words, involves a thousand choices, but not a billion, because most of the big choices have already been made. These frozen choices are what gives institutions their vitality — they are in fact what make them institutions. Freed of the twin dangers of navel-gazing and random walks, an institution can concentrate its efforts on some persistent, medium-sized, and tractable problem, working at a scale and longevity unavailable to its individual participants.

Institutions also reduce the choices a society has to make. In the second half of the 20th century, “the news” was whatever was in the newspaper on the morning, or network TV at night. Advertisers knew where to reach shoppers. Politicians knew who to they had to talk to to get their message out (sometimes voluntarily, sometimes not.) Readers understood an Letters page as the obvious way of getting wider circulation for their views.

That dual reduction of choices masks an essential asymmetry, though. Institutions are designed to reduce they choices for their members, but they only happen to reduce the choices in society. A publisher may want reporters at their desks at 10 am, and to be the main source of breaking news for the paper’s readers. The former desire is under the publisher’s control; the latter not.

Now this seems easy enough to describe, but people inside institutions tend to confuse the two, to believe their institution is in control of both their daily routine and their destiny. And of course, the longer any given institution maintains a stable role, the less that role seems like an accident and the more it seems like a robust feature of reality.

* * *

Starkman and I could, I think, agree on this story so far, and indeed, most of the time institutional vitality is a safe bet. The ability of institutions to adapt slowly while preserving continuity of mission and process is exactly what lets them last longer than a single leader or lifespan. When change in the outside world outstrips an institution’s adaptive capabilities, though, the ability to defend the internal organization from outside pressures can become a liability. Stability can tun into rigidity and even institutional blindness.

Newspapers were stable for a really long time. The 1830s saw the rise of the penny press, when advertisers generated enough subsidy for the cover price to be reduced from six cents to one, with an attendant explosion in readership. Improbably enough, that model survived for most of two centuries with little more than tweaks and updates, right up to the sudden increase in freedoms occasioned by the internet, freedoms enjoyed by both advertisers and readers. The erosion of geographic sinecure made every story available everywhere. Casual aggregation from multiple sources delighted readers but eroded loyalty. Competition turned paywalls into cul-de-sacs instead of toll roads. Upstart web services (Yahoo and iVillage, Slate and Salon) first set then compressed a market price for ads that the newspapers never had a chance to alter. Users forwarding stories to one another increased audience size, but destroyed scarcity.

Despite these challenges to newspapers, Starkman believes that we can and must “…find ways to preserve and transfer their most important attributes to a digital era, even as we push them to adapt to new financial, technological, and cultural realities.” I don’t believe we must do this, because I don’t believe we can do this. That, I think, is the core diference between our views.

There are thousands of papers in the US, all facing simultaneous pressures on their revenues, their organizational form, the loyalty of their readership, the ferocity of the competition, and even their sense of self. If, seeing those challenges, we FONeros still thought it was possible to preserve most papers in something like their current form, we’d adopt Plan A too. You’d only pursue something like Plan B if you’d given up on broad institutional preservation in the first place.

* * *

To my eye, Starkman stacks the deck when comparing Plans A and B. He lists only three examples of successful “FON” journalism (US Attorney firings, “macacca,” and “bittergate“), but his recounting of the glories of print go back to Ida Tarbell. Tarbell was indeed a terrific reporter, but her byline has been somewhat scarce of late, given she’s been dead 70 years. Comparing a 5 year stretch of recent experiments with the greatest hits of newspapering since the McKinley administration may rally the home team, but it doesn’t make for a particularly informative comparison.

Like a Yeats of the newspaper world, Starkman yearns for the restoration of a culture considerably purer than the actual newspaper business has ever been. Reading Confidence Game, you’d never know that most papers are not like the NY Times, that most of what appears in their pages is syndicated, that sports is often better represented on the masthead than hard news. You’d never know that more American papers printed today will include a horoscope than international news. You’d never know that newspapers are institutions where grown men and women are assigned to write stories about dogs catching frisbees.

Saying newspapers will provide a stable home for reporters, just as soon as we figure out how to make newspapers stable, is like saying that if we had some ham, we could have a ham sandwich, if we had some bread. We need to support the people who cover hard news, but when you see a metro daily for a town of 100,000 that employs only six such reporters (just 10% of the masthead, much less total staff), saving the entire edifice just to support that handful looks a lot harder than just finding new ways to support them directly.

This view is, as Starkman says, anti-institutional, starting as it does from the premise that the outside world is changing faster than most newspapers’ adaptive capabilities. They have responded to 20 consecutive quarters ad revenue decline with the evisceration of international desks and newsroom staff. More is on the way. No medium has ever survived the indifference of 25 year olds.

* * *

We have long since entered a period of hybridization — William Bastone quitting The Village Voice to found Smoking Gun and Nate Silver selling fivethirtyeight.com to the NY Times, Andy Carvin and Anjali Mullany pioneering new forms of live coverage from inside traditional organizations, Amanda Michel’s career implementing citizen journalism at Huffington Post, then at ProPublica, then at The Guardian, these are concrete demonstrations that the old dichotomies of traditional vs. new media, professionals vs. amateurs, incumbents vs. insurgents, have long since stopped being real, hard choices, and have instead become points on an increasingly traversable spectrum.

All of this seems to offer the grandmotherly option between Starkman and the FON crew — “You’re both right, dear. We need institutions and we need experiments.” Even given this hybridization, though, our views diverge: Plan A assumes that experiments should be spokes to the newspapers’ hub, their continued role as the clear center of public interest journalism assured, and on the terms previously negotiated.

Plan B follows Jonathan Stray’s observations about the digital public sphere: in a world where Wikipedia is a more popular source of information than any newspaper, maybe we won’t have a clear center anymore. Maybe we’ll just have lots of overlapping, partial, competitive, cooperative attempts to arm the public to deal with the world we live in.

Some of the experiments going on today, small and tentative as they are, will eventually harden into institutional form, and that development will be as surprising as the penny press subsidizing journalism for seven generations. The old landscape had institutions and so will the new one, but this doesn’t imply continuity. We still have companies called Western Union and ATT, but as the communications landscape changed, they have become almost unrecognizably different from their former selves. Likewise, as the presses fall silent over the next ten years, even papers that survive will see their internal organization and their place in the ecosystem altered beyond our ability to predict.

If you believed, as Starkman clearly does, that this view is not just incorrect but odious, that the current form of the newspaper remains a good general fit for public interest journalism, merely going through a rough patch, then you’d be eager to dial down the ‘try anything’ ethic in favor of the hard, grinding work of rebuilding and shoring up the institutions that have served us so well these last several decades.

But if you believe, as I do, that many of those institutions are so mismatched to the task at hand that most of them face a choice, at best, between radical restructure and outright collapse, well, in that case, you’d probably find the smartest 25 year olds you know, and try to convince them that now would be a pretty good time to start working on Plan B.

Why We Need the New News Environment to be Chaotic

July 9, 2011

The business environment for newspapers continues to be grim. Pew recently reported that advertising revenue rebounded in 2010 for all forms of media, except newspapers.* This might just be a matter of transitioning from print to digital revenues but for the fact that the market values a print reader far more than a digital one. The more or less official label for this problem is “analog dollars to digital dimes”; because of the enormous difference in assumed value per reader, lost value from print is not made up for by gains in digital readership.

The ‘analog dollars to digital dimes’ problem doesn’t actually seem to be a problem. It seems to be a feature of reality. Digital revenue per head is not replacing lost print revenue and, barring some astonishment in the advertising market, it never will. There is no supply-side scarcity to boost margins; power over aggregation has moved from producer to consumer; advertisers prefer selling cameras and shoes on purpose-built sites; readers feel the same about finding jobs and dates; and there are few commercial or geographic restraints on competition. The market values analog readers more highly than digital ones because the market is right.

Seeing this, several people have started looking for ways to exit that market. One of the most widely circulated of these ideas is David Swenson and Michael Schmidt’s proposal that newspapers be subsidized the way colleges or foundations are.* Steve Coll followed this with a suggestion that commercial papers be allowed to reinvent themselves as non-profits.*

The intuition common to such proposals is that advertising revenues can no longer be relied on to fund serious reporting. (Gadget reviews and celebrity coverage, yes, but not the police beat.) Evidence entered for this thesis includes the decline in print’s commercial fortunes*, the attendant downsizing of newsrooms—already 30% smaller, on average, than a decade ago*—and the outright collapse of papers like the Rocky Mountain News, Albuquerque Post, and Cincinnati Post.*

Proponents of market-supported journalism, among them Jack Shafer*, Alan Mutter*, and Jeff Jarvis*, have replied to these non-profit proposals at length, but their common thesis, distilled to its essence, is “Oh please.” This group points out that there is simply not enough philanthropic money to support current news-gathering organizations, and any suggestion that there someday might be isn’t strategy but fantasy.

Evidence for this thesis includes the gap between supply and demand for philanthropic funding* or from the narrowness of funding sources, as with ProPublica’s getting 80 times as much money from its board as from online donations.*

This argument between Journalism as Philanthropy and Journalism as Capitalism (to borrow Jarvis’s phrase) seems like a dilemma, in the literal sense of the word—a forced choice between two alternative premises. It isn’t really, though, because there is also the grandmotherly option: Both groups could be right. It’s possible that for-profit revenue is shrinking irreversibly and that non-profit funding sources won’t make up for the shortfall. The least we can say about this possibility is that it can’t be discounted on current evidence.

* * *

One proposed response is to radically reform newspapers as both organizations and businesses. Here, as one of countless examples, is something from a recent post by Tom Matlack*, the former CFO of The Providence Journal:

[T]here is no longer an uncomfortable ménage à trois between newspapers, readers and advertisers. It’s monogamy at last. Newspapers have to focus all their energy on producing extraordinary journalism—in a form and in substance—which thrills us as consumers. [...] So fear not. Your paper is not dead. You just have to demand, and be willing to pay for, a news product worthy of your affections.

Matlack’s sentiment is clearly heartfelt, and creating a high-quality product for a group of loyal and passionate readers willing to pay for it certainly sounds like an interesting business to get into. It just doesn’t sound like the newspaper business.

Here’s what the newspaper business sounds like: the modestly talented son of the founder can generate double-digit margins based on little more than the happy accident that there are people who like football and buy cars living within 30 miles of his house.

That’s the newspaper business, or at least it was until recently. The average US paper runs more soft than hard news, uses more third-party content than anything created by their own staff, and reaches more people who care about local teams than local zoning. Telling the publishers of those papers to create a digital product so extraordinary that readers will pay full freight is a tacit admission that they do not know how to make such a product today.

Much public worry about newspapers concerns a relative handful of excellent dailies with national or international ambitions. Most papers, however, aren’t like that. The New York Times and the Enid, Oklahoma News and Eagle occupy different parts of the news ecosystem, and they face different stresses and fates, but more papers—many more—exist at the News and Eagle end of the spectrum.*

It’s easy to view the current business climate as a culling of the herd, but newspapers are not barbershops; the closing of a hair-cutting business in Oklahoma wouldn’t much affect people in New Hampshire, but the closing of a paper would, because the nation’s news publications are sewn together in a crazy quilt of shared cost and effort.

The News and Eagle, the Laconia Citizen, the Biloxi Sun-Herald, the Deer Park Tribune, and a thousand more such papers are all but monopoly suppliers of local news, they all train young reporters who go on to work elsewhere, they all employ the stringers who are on the scene when a tornado hits, and they all buy syndicated content from the Associated Press or King Features, who in turn lower costs for some publishers while raising revenue for others. When a paper fires reporters or closes outright, it further weakens that fabric.

* * *

Buy a newspaper. Cut it up. Throw away the ads. Sort the remaining stories into piles. Now, describe the editorial logic holding those piles together.

If you’ve picked a general interest paper, this will be hard. I recently learned, from a single day’s paper, that a bombing in Kirkuk killed 27, that Penelope Cruz has only good memories of filming Pirates of the Caribbean while pregnant, that many U.S. business hotels are switching to ‘shower-only’ bathrooms, and that 30-year fixed mortgages fell from 4.63% to 4.61% the week before.

The rationale for creating such a bundle went something like this: “We will print enough content to fill the hole left after we’ve sold the advertising space. We will include content proportional to the amount and intensity of reader interest, modified somewhat by editorial judgment. Overall, the value of the bundle will be more than the sum of its parts.”

Many people who worked for newspapers didn’t think of this as something that happened to work well in certain cities and towns during a particular era, but as something real: “Of course Ratko Mladic’s arrest belongs in the same package as a photo spread on ornamental grasses! How else would you do it?” For all that selling such a bundle was a business, though, people have never actually paid for news. We have, at most, helped pay for the things that paid for the news.

So long as newspapers faced little competition for advertisers or readers, this was a distinction without a difference, but as papers are being sundered by the internet, we can see how tangled the system always was. Outside a relative handful of financial publications, there is no such thing as the news business. There is only the advertising business. The remarkable thing about the newspapers’ piece of that business isn’t that they could reliably generate profits without accomplishing much in the way of innovation—that could just as easily describe the local car dealership. The remarkable thing is that over the last couple of generations, those profits supported the fractional bit of those enterprises that covered the news.

This subsidy relied on cultural logic peculiar to newspapers; publishers were constrained not just by their investors but by their editors (who expected the paper to be ethical in the short term) and by their families (who expected the paper to be viable over the long term). In return, a publisher could extract some of the value of the paper in prestige and sinecure instead of cash.

This system was never ideal—out of the crooked timber of humanity no straight thing was ever made—and long before Craig Newmark and Arianna Huffington began their reign of terror, Gannett and Scripps were pioneering debt-laden balance sheets, highly paid executives, and short-term profit-chasing. But even in their worst days, newspapers supported the minority of journalists reporting actual news, for the minority of citizens who cared. In return, the people who followed sports or celebrities, or clipped recipes and coupons, got to live in a town where the City Council was marginally less likely to be corrupt.

Writing about the Dallas Cowboys in order to take money from Ford and give it to the guy on the City Desk never made much sense, but at least it worked. Online, though, the economic and technological rationale for bundling weakens—no monopoly over local advertising, no daily allotment of space to fill, no one-size-fits-all delivery system. Newspapers, as a sheaf of unrelated content glued together with ads, aren’t just being threatened with unprofitability, but incoherence.

* * *

This fall, I’m joining NYU’s journalism program, where, for the first time in a dozen years, I will teach undergraduates. Someone who turns 19 this year will have not one adult memory of the 20th century; for them, the Contract With America, the Monica Lewinsky scandal and the first Gulf War are roughly contemporaneous events, just as, for my 19 year old cohort, the Summer of Love, the Watts’ riots, and Kent State all seemed to have happened in that one busy month we called The 60s. When it comes time to explain the media landscape of the 20th century, I will be teaching my own youth as ancient history.

I could tell these students that when I was growing up, the only news I read was thrown into our front yard by a boy on a bicycle. They might find this interesting, but only in the way I found it interesting that my father had grown up without indoor plumbing. What 19 year olds need to know isn’t how it was in Ye Olden Tymes of 1992; they need to know what we’ve learned about supporting the creation and dissemination of news between then and now. Contemplating what I should tell them, there are only three things I’m sure of: News has to be subsidized, and it has to be cheap, and it has to be free.

News has to be subsidized because society’s truth-tellers can’t be supported by what their work would fetch on the open market. However much the Journalism as Philanthropy crowd gives off that ‘Eat your peas’ vibe, one thing they have exactly right is that markets supply less reporting than democracies demand. Most people don’t care about the news, and most of the people who do don’t care enough to pay for it, but we need the ones who care to have it, even if they care only a little bit, only some of the time. To create more of something than people will pay for requires subsidy.

News has to be cheap because cheap is where the opportunity is right now. For all that the Journalism as Capitalism people can sound like Creflo Dollar mid-sermon, they are right to put their faith in new models for news. If for-profit revenue is shrinking and non-profit funding won’t make up the shortfall, we need much cheaper ways of gathering, understanding, and disseminating news, whether measured in information produced or readers served.

And news has to be free, because it has to spread. The few people who care about the news need to be able to share it with one another and, in times of crisis, to sound the alarm for the rest of us. Newspapers have always felt a tension between their commercial and civic functions, but when a publication drags access to the news itself over to the business side, as with the paywalls at The Times of London or the Tallahassee Democrat, they become Journalism as Luxury. In a future dominated by Journalism as Luxury, elites would still get what they need (a tautology in market economies), but most communities would suffer; imagine Bell, California times a thousand, with no Ruben Vives to go after the the politicians.*

The thing I really want to impress on my students is that the commercial case for news only matters if the profits are used to subsidize reporting the public can see, and that civic virtue may be heart-warming, but it won’t keep the lights on, if the lights cost more than cash on hand. Both sides of the equation have to be solved.

* * *

Real news—reporting done for citizens instead of consumers—is a public good. This is true both in the colloquial sense of ‘good for the public’ and in the economic sense of ‘best provisioned for a whole group at once.’

The supplier of last resort for public goods is usually the government, but in the United States, public funding of media has always been politically fraught, outside a few subsidies like reduced postal rates. This leaves us the problem of producing a public good without much in the way of public monies.

Taking cash from advertisers is one way to do this, though a less good one than it used to be. As Jay Rosen points out*, many other ways are possible: NewWest.net gets money from its conference business*; The Guardian from the Scott Trust*. Donations are still another: some organizations have a syndicate of large donors, as with ProPublica * or The American Independent*. Others have many small donors, as with the crowdfunding of Spot.us projects, or the listener donations to NPR.

And, critically, subsidy can be in savings rather than cash. Some of what professionals did in the old model can now be done in combination with amateurs, or crowds, or machines: MAPlight* and PoliGraft* and Sunlight’s Lobbying Tracker* couldn’t track links between money and politics without online databases; the Charlotte News Alliance* and the Tuscon Citizen * rely on local bloggers; the Davis Wiki* and the the Oil Spill Crisis Map* provide structure to user-contributed material; Tackable is betting that the first photographer on the scene will be a citizen with a phone*.

None of the models being tried today are universally adoptable; the most we can say is that each of them happens to work somewhere, at least for the moment. This may seem like weak tea, given the enormity of the current changes, but if our test for any new way of producing news is whether it replaces all the functions of a newspaper, we’ll build things that look like newspapers, and if replicating newspapers online were a good idea, we wouldn’t be in this mess in the first place.

If we adopt the radical view that what seems to be happening is actually happening, then a crisis in reporting isn’t something that might take place in the future. A 30% reduction in newsroom staff, with more to come, means this is the crisis, right now. Any way of creating news that gets cost below income, however odd, is a good way, and any way that doesn’t, however hallowed, is bad.

Having one kind of institution do most of the reporting for most communities in the US seemed like a great idea right up until it seemed like a single point of failure. As that failure spreads, the news ecosystem isn’t just getting more chaotic, we need it to be more chaotic, because we need multiple competing approaches. It isn’t newspapers we should be worrying about, but news, and there are many more ways of getting and reporting the news that we haven’t tried than that we have.

Free speech Is Not A Pony

June 12, 2011

I’m quoted in the New York Times today talking about the difficulty in the State Department’s attempt to portray freedom of speech as a human rights issue, while sidestepping the fact that freedom of speech is a destabilizing force in autocratic governments:

You can’t say ‘All we want is for people to speak their minds, not bring down autocratic regimes’ — they’re the same thing.

“Thing” was a sloppy word to use; what I should have said is that they are the same desire. The State Department can’t say they want people in autocracies to have freedom of speech without admitting that this is an expressly political desire–as are all assertions about human rights, however construed–and that this desire is expressly inimical to autocracy.

The next paragraph of the Times’ piece adds just such a clarification:

He added that the United States could expose itself to charges of hypocrisy if the State Department maintained its support, tacit or otherwise, for autocratic governments running countries like Saudi Arabia or Bahrain while deploying technology that was likely to undermine them.

Freedom of speech is inimical to autocratic control, both as cause and effect. When people are free to publicly contradict assertions made by their government, it makes it harder for those governments to govern unilaterally, and the kind of governments that commit themselves to freedom of speech are usually (perhaps invariably?) democracies.

My friend Zeynep Tufekci of U-Maryland and technosociology.org summed up this idea far better than I did: “The internet isn’t your pony such that you can support free speech and expect it not to have political consequences.”

After Tahrir: Protecting US Muslims from Bigotry

March 24, 2011

Of all the images to come out of the North African uprisings, this is the one that consistently makes me tear up.

These are Coptic Christians in Cairo’s Tahrir Square, holding hands around Muslims during Friday prayer, to protect them from surprise attack.

We could do this. We could do this here, in the United States, to keep our Muslim citizens safe. All of us, Christians and Jews and atheists, anyone committed to the First Amendment, to the idea that religious freedom is essential to American society, could join hands to protect Muslims living here from rising prejudice.

The people of the United States are capable of visiting horrors on those we decide aren’t really like “us” enough to count: Blacks owned, women silenced, Jews segregated, gays repressed, Japanese interned, the catalog of American hate and fear runs to many pages.

And now it’s happening again, with political fear-mongering around the Park51 Islamic community center in New York and Rep. Peter King’s recent grandstanding on radicalization in the American Muslim community fueling a ‘guilty until proven innocent’ attitude towards Muslim citizens and organizations. We should certainly worry about murderous homegrown radicals, but that’s because murderous radicals are dangerous, not because they are religious; David Koresh, Timothy McVeigh, and David Brian Stone were all homegrown and all evil, but none of them were Muslim.

I don’t know how bad this kind of prejudice will get, but it will get worse than today. To counter this threat, we have to integrate the Muslim experience into the American experience, just as the Protestants came to shed anti-Catholic prejudice, and the Christians anti-Jewish prejudice. As imperfect as those expansions have been, even today, the United States is a better place for having widened our sense of who constitutes “us,” of having engaged in what Richard Rorty called justice as larger loyalty.

I don’t know how to do this, how to keep people safe from the bigots who want the very fact of Muslim faith or heritage to generate suspicion and oppression. I do know some ways to start. We can respond to prejudice when we hear it; the thing that woke me up was hearing one of my cousins argue that a mosque in lower Manhattan was an affront to American identity, as if Muslim citizens were inherently less American than Christian ones.

We can care about how Muslims are treated, here and abroad. Democratic governments respond to public preferences; bigotry thrives on the silence of those of us who support freedom of religion, but not out loud. (Hence this post.) Democratic governments also respond to voters; it’s a safe bet that the organizations helping register Muslim-American voters in 2008 will do so again in 2012, and they need our help.

We can donate. I’ve given money to the Muslim Public Affairs Council, and to Irshad Manji’s Project Ijtihad (ijtihad meaning, roughly, independent reasoning.) These are of course political choices — Project Ijtihad is explicitly progressive and feminist — and your politics may run more to What Unites Us, an organization set up to combat anti-Muslim prejudice, to the American Muslim Law Enforcement Officers Association, or any number of local or interfaith organizations opposed to religious bigotry.

Perhaps most importantly, we can tell people we’ve donated or participated. It will be better to donate $5 to a group combatting anti-Muslim prejudice and then announce it on Facebook or Twitter, than to give $500 quietly. (I’m using #AfterTahrir.)

As President Bush said at the Islamic Center of Washington

Those who feel like they can intimidate our fellow citizens to take out their anger don’t represent the best of America, they represent the worst of humankind, and they should be ashamed of that kind of behavior.

Bush was right. America’s finest hours have always been integrative and our most shameful ones prejudiced. Now, again, we have to pick, and we have an opportunity to learn from the protesters in Tahrir what political courage looks like.

Consulting with Libya in 2007

March 1, 2011

Over the weekend, Evgeny Morozov asked about my consulting with the Libyan government in 2007. In March of that year, I was invited by Monitor Consulting to come to Boston to speak to a Libyan IT minister about using social software to improve citizen engagement in coastal towns. The idea was that those cities would be more economically successful if local policies related to the tourist trade were designed by the locals themselves. (I talked about that work briefly in this interview [28:30-30:30])

After the meeting in Boston, the imagined project didn’t materialize. I presume this was because the government choked on the inability to get the economic effects of citizen participation without getting the political effects as well, an idea they claimed to want but in fact didn’t.

Morozov also suggests that “anyone who thought they could democratize Libya via Wikis needs to make a public apology.” This is the same caricature he often uses, implying that people like me who think that social tools can improve outcomes actually believe that tools cause those outcomes. What we believed at the time was that Libya’s planned devolution of political power to individual towns was real; what we learned was that it wasn’t. I’m sorry I wasn’t able to help expand representative government in Libya, but I’m not sorry I tried.

I am, in other words, exactly as naive as Morozov accuses me of being. As I’ve said elsewhere, the best reason to believe that social media can aid citizens in their struggle to make government more responsive is that both citizens and governments believe that.

Half-formed thought on Wikileaks & Global Action

December 31, 2010

This is a coda to my earlier post, Wikileaks and the Long Haul. It’s an attempt to express a partially formed thought about the Pentagon Papers case and the global media environment.

A bit of potted history first: The Pentagon Papers were a secret history of US involvement in Vietnam, produced by the Department of Defense and leaked in 1971 by Daniel Ellsberg to the NY Times, who published excerpts and analysis from them. The government attempted to prosecute the Times under the Espionage Act; the Times, with Floyd Abrams as their lead attorney, argued the case before the Supreme Court. The Times won, and the decision, New York Times Co. v. United States (403 U.S. 713), established the principle that it was illegal to leak secrets, but not to publish leaks. Justice Hugo Black, writing for the majority, said “Only a free and unrestrained press can effectively expose deception in government.”

I was thinking about Black’s opinion, and particularly his emphasis on an “unrestrained press”, in light of two things I’ve read on Wikileaks. The first, and more recent, is Floyd Abrams tortured attempt to deny that his winning arguments in the Pentagon Papers case set the operative precedent for Wikileaks.

His essay is an attempt to square two things he clearly believes deeply — Wikileaks is bad and must be stopped, and that its actions may not be illegal. He doesn’t want to believe that second thing, but as one of the pre-eminent litigators of 1st Amendment law, he also understands what the precedents say, and glumly comes to this conclusion:

[I]f Mr. Assange were viewed as simply following his deeply held view that the secrets of government should be bared, notwithstanding the consequences, he might escape legal punishment.

Abrams is upset enough about Wikileaks (and Assange) that he can’t bring himself to describe the clear meaning of this conclusion: under the Pentagon Papers precedent, Assange might “escape legal punishment” the ordinary way, by being innocent of having committed a crime.

This in turn brought to mind my NYU colleague Jay Rosen’s observation last summer, during the Afghan War logs leak, that Wikileaks is the first stateless news organization:

Appealing to national traditions of fair play in the conduct of news reporting misunderstands what Wikileaks is about: the release of information without regard for national interest. In media history up to now, the press is free to report on what the powerful wish to keep secret because the laws of a given nation protect it. But Wikileaks is able to report on what the powerful wish to keep secret because the logic of the Internet permits it.

So here’s the half-formed thought: Abrams is wrong that the Pentagon Papers case isn’t the obvious precedent, but he’s right that Wikileaks is so different that the meaning of that case isn’t clear in the present situation.

* * *

To draw a distinction between the Pentagon Papers precedent and what Wikileaks means, I’m going to use slightly different language than Rosen’s notion of statelessness. (I think Wikileaks is less stateless than it is multi-homed, allowing it some freedom from traditional ‘single point of censorship’ problems that plague other international media).)

Let me propose, for the sake of argument, two labels for action that spans more than one country: international, and global. International actors are actors rooted in a nation, even when they are able to participate in activities all over the world, while global actors are unrooted; global actors have, as their home environment, the globe.

The closer an activity comes to attaining the condition of pure information, the more global it can be. By way of analogy, the LSD business is more global than the cocaine business, because coca leaves only grow in certain climates, but lysergic acid can be synthesized anywhere. Media is like this as well: The internet is more global than the telephone network, even though both systems can send data between any two points in the world. Similarly, Wikileaks is more global than the BBC or Al Jazeera; those organizations are very large, but they are international, with a home base as rooted in a particular place as a coca farmer is.

The most dramatic of Wikileaks’ breaks with previous journalism is the global nature Rosen identified. The biggest difference between the Pentagon Papers case and Wikileaks is not the legal precedent, but the fact that the Pentagon Papers case was an entirely national affair.

Though some of the most impassioned and vocal participants in the Vietnam war protest movement genuinely cared about the fate of the Vietnamese, the bulk of the participants were animated by a much more proximate goal: ending the draft in the US. The publication of the Pentagon Papers did not seem to have helped end the war, but exposure of the military’s frank internal assessment of the compromised nature of the conflict made it harder to ask middle-class parents to sacrifice their children to that kind of action. (It also helped feed into the collapse of trust in the government and of authority figures generally.)

All this was in the US context, as was every actor involved: Ellsberg, Abrams, the NY Times, the Pentagon, the Supreme Court, the protesting citizens, and so on. No one was out of the reach of the Federal Government, and the effect of the Papers’ publication, in a US new outlet, was a family affair.

Wikileaks has been global from the beginning, and the additional complexity of both jurisdiction and extradition make this particular problem much much more complex than any issues, legal or practical, triggered by the Pentagon Papers. Wikileaks has been operating since 2006, the military has regarded it as a significant threat since at least 2008, and the US Attorney General still has difficulty framing charges he thinks he can win.

* * *

For many of our most important social systems, we resolve clashing principles by providing an escape valve, in the form of a set of actors who are less rule-bound than the rest of the system. The most famous and ancient is the jury, a collection of amateurs who can, in the face of clear laws and evidence, simply not return the verdict a judge would have returned.

So with secrecy. Though I am not a lawyer, the Supreme Court’s 1971 ruling seems to say that there is no law-like way to balance the State’s need for secrets with the threats secrets pose to democracies, so it simply said that the 1st Amendment provides immunity to publishers in most circumstances; the presence of publishers as “unrestrained actors” provides one of the many limits on government power that make democracies work.

This immunity sets up publishers as self-regulating checks to government power, albeit in a system that can never be made intellectually coherent — neither total success nor total failure of the government to keep secrets would protect the United States as well as a regime of mostly success with periodic, unpredictable failures.

The Pentagon Papers decision says the government faces a “high bar” to proving that a publisher’s actions are illegal. What’s legal for publishers is thus whatever keeps them under this high bar, but no one knows where this bar is at any given moment, except that it is high enough that no publisher has ever been successfully prosecuted for espionage.

But here’s where Black was wrong, or at least only partly right: the press in the US isn’t unrestrained, just because of this high bar. Instead, the US press is self-restrained. Our curious system of outlawing certain kinds of speech — libel, release of trade secrets, and so on — while also largely forbidding the government from prevention of publication (prior restraint) or creating a climate of fear and doubt for publishers (chilling effects) would seem impossible to balance within a legal system, but that’s because the balance is produced by extra-legal constraints.

A publisher is (or was, in 1971) a commercial, nationally-rooted media firm subject to significant tradeoffs between scale and partisanship. Large publishers had to be deeply embedded in the culture they operated in, and they had to reflect local mainstream views on most matters most of the time, to find and retain both revenue and audience. Fans of game theory will recognize these conditions as those required for an iterated game of the Prisoner’s Dilemma, where the press exhibits self-restraint from short term defection against the US’s interests, in order to benefit from an amicable relationship with the government over the long haul.

This is why the difference between the Times, as an international actor, and Wikileaks, as a global one, matters so much. Wikileaks does not have to play an iterated game of Prisoner’s Dilemma with the US. Not only is Wikileaks not housed in the US, it isn’t housed in any single other nation the US could complain to. They can defect at will. (You can always burn a partner in Prisoner’s Dilemma if they can’t get back at you.) Wikileaks hasn’t defected all the way, of course, releasing only ~2000 of the ~250,000 cables they have; the point is that this restraint is not forced on them by the US.

The legal bargain from 1971 simply does not and cannot produce the outcome it used to. This is one of the things freaking people in the US government out about the long-term change in the media environment — not that the law has changed, but that the world has. Industrial era law, applied to internet-era publishing, might allow for media outlets which exhibit no self-restraint around national sensitivities, because they are run by people without any loyalty to — or, more importantly, need of — national affiliation to do their jobs.

* * *

There is, in much of the commentary about Wikileaks online, a kind of echo of Black’s 1971 opinion, a fantasy of unrestrained action. This comes in two flavors: one group believes that since Julian Assange is not a US citizen and not operating on US soil, the US could never charge him with anything. These people seem not to have heard of extradition agreements; the legal system is at least international enough to allow charges to be brought against foreign nationals and have them tried in the US. Whether that will happen to Julian I don’t know, but it’s certainly not out of the realm of the possible.

The other ‘unrestrained action’ group believes that, as Assange is not a US citizen, there is nothing to stop us from simply finding him and having him killed. This group seems not to recognize that political murders are regarded as something of a no-no by our allies, and that such a move would not exactly be a net win for the US, as well as potentially getting the agents of such an illegal approach arrested and tried.

Neither of these fantasies is going to come true. The US cannot simply remove Assange with no consequences, nor is he automatically cleared from another government’s decision to extradite him to the US, if the Attorney General charges him. This is going to take time and effort to work itself out.

Society is made up of competing goods that can’t be resolved in any perfect way — freedom vs. liberty, state secrets vs. citizen oversight — but the solutions to those tensions always take place in a particular context. Sometimes a bargain is so robust it lasts for centuries, as with trial by jury, but sometimes it is so much a product of its time that it does not survive the passing of its era.

I think that this latter fate has befallen our old balance between secrets and leaks. This does not mean that the Pentagon Papers precedent shouldn’t free Wikileaks from prosecution, but it does mean the old rules will not produce the old outcomes. In another difference with the Pentagon Papers case, Bradley Manning (or the person who copied the cables from SIPRnet) has gotten very little attention compared to Wikileaks, even though, in both law and practice, he is clearly the person most culpable. Like the music industry, the government is witnessing the million-fold expansion of edge points capable of acting on their own, without needing to ask anyone for help or permission, and, like the music industry, they are looking at various strategies for adding control at intermediary points that were left alone under the old model.

Julian claims that the history of these matters will be divided into “pre-” and “post-Cablegate” periods. This claim is grandiose and premature. However, it is not, on present evidence, visibly wrong.

It’s possible that the plain meaning of the Pentagon Papers case will clear Assange and Wikileaks, full stop, and the era of self-restraint of the press in response to extra-legal constraints is over, at least in the US context. It’s possible that the Pentagon Papers case will be re-adjudicated, and the press freedoms of the traditional press in the US will be dramatically constrained, relative to today. It’s possible that new laws will be written by Congress; it’s possible that those laws will be vetoed, or overturned, or amended. Whatever happens, though, this is new ground, and needs to be hashed out as an exemplar of the clash of basic principles that it is.

Wikileaks and the Long Haul

December 6, 2010

Like a lot of people, I am conflicted about Wikileaks.

Citizens of a functioning democracy must be able to know what the state is saying and doing in our name, to engage in what Pierre Rosanvallon calls “counter-democracy”*, the democracy of citizens distrusting rather than legitimizing the actions of the state. Wikileaks plainly improves those abilities.

On the other hand, human systems can’t stand pure transparency. For negotiation to work, people’s stated positions have to change, but change is seen, almost universally, as weakness. People trying to come to consensus must be able to privately voice opinions they would publicly abjure, and may later abandon. Wikileaks plainly damages those abilities. (If Aaron Bady’s analysis is correct, it is the damage and not the oversight that Wikileaks is designed to create.*)

And so we have a tension between two requirements for democratic statecraft, one that can’t be resolved, but can be brought to an acceptable equilibrium. Indeed, like the virtues of equality vs. liberty, or popular will vs. fundamental rights, it has to be brought into such an equilibrium for democratic statecraft not to be wrecked either by too much secrecy or too much transparency.

As Tom Slee puts it, “Your answer to ‘what data should the government make public?’ depends not so much on what you think about data, but what you think about the government.”* My personal view is that there is too much secrecy in the current system, and that a corrective towards transparency is a good idea. I don’t, however, believe in total transparency, and even more importantly, I don’t think that independent actors who are subject to no checks or balances is a good idea in the long haul.

If the long haul were all there was, Wikileaks would be an obviously bad thing. The practical history of politics, however, suggests that the periodic appearance of such unconstrained actors in the short haul is essential to increased democratization, not just of politics but of thought.

We celebrate the printers of 16th century Amsterdam for making it impossible for the Catholic Church to constrain the output of the printing press to Church-approved books*, a challenge that helped usher in, among other things, the decentralization of scientific inquiry and the spread of politically seditious writings advocating democracy.

This intellectual and political victory didn’t, however, mean that the printing press was then free of all constraints. Over time, a set of legal limitations around printing rose up, including restrictions on libel, the publication of trade secrets, and sedition. I don’t agree with all of these laws, but they were at least produced by some legal process.

Unlike the United States’ current pursuit of Wikileaks.*

I am conflicted about the right balance between the visibility required for counter-democracy and the need for private speech among international actors. Here’s what I’m not conflicted about: When authorities can’t get what they want by working within the law, the right answer is not to work outside the law. The right answer is that they can’t get what they want.

The Unites States is — or should be — subject to the rule of law, which makes the extra-judicial pursuit of Wikileaks especially nauseating. (Calls for Julian’s assassination are even more nauseating.) It may be that what Julian has done is a crime. (I know him casually, but not well enough to vouch for his motivations, nor am I a lawyer.) In that case, the right answer is to bring the case to a trial.

In the US, however, the government has a “heavy burden”, in the words of the Supreme Court, for engaging in prior restraint of even secret documents, an established principle since New York Times Co. vs. The United States*, when the Times published the Pentagon Papers. If we want a different answer for Wikileaks, we need a different legal framework first.

Though I don’t like Senator Joseph Lieberman’s proposed SHIELD law (Securing Human Intelligence and Enforcing Lawful Dissemination*), I do like the fact that it is a law, and not an extra-legal avenue (of which Senator Lieberman is also guilty.*) I also like the fact that the SHIELD Law makes it clear what’s at stake: the law proposes new restraints on publishers, and would apply to the New York Times and The Guardian as it well as to Wikileaks. (As Matthew Ingram points out, “Like it or not, Wikileaks is a media entity.”*) SHIELD amounts to an attempt to reverse parts of New York Times Co. vs. The United States.

I don’t think such a law should pass. I think the current laws, which criminalize the leaking of secrets but not the publishing of leaks, strike the right balance. However, as a citizen of a democracy, I’m willing to be voted down, and I’m willing to see other democratically proposed restrictions on Wikileaks put in place. It may even be that whatever checks and balances do get put in place by the democratic process make anything like Wikileaks impossible to sustain in the future.

The key, though, is that democracies have a process for creating such restrictions, and as a citizen it sickens me to see the US trying to take shortcuts. The leaders of Myanmar and Belarus, or Thailand and Russia, can now rightly say to us “You went after Wikileaks’ domain name, their hosting provider, and even denied your citizens the ability to register protest through donations, all without a warrant and all targeting overseas entities, simply because you decided you don’t like the site. If that’s the way governments get to behave, we can live with that.”

Over the long haul, we will need new checks and balances for newly increased transparency — Wikileaks shouldn’t be able to operate as a law unto itself anymore than the US should be able to. In the short haul, though, Wikileaks is our Amsterdam. Whatever restrictions we eventually end up enacting, we need to keep Wikileaks alive today, while we work through the process democracies always go through to react to change. If it’s OK for a democracy to just decide to run someone off the internet for doing something they wouldn’t prosecute a newspaper for doing, the idea of an internet that further democratizes the public sphere will have taken a mortal blow.

The Times’ Paywall and Newsletter Economics

November 8, 2010

It is, perhaps, the end of the beginning.

In early July, Rupert Murdoch’s News Corporation placed its two London-based “quality” dailies, the Times and Sunday Times, behind a paywall, charging £1 for 24 hours access, or £2 a week (after an introductory £1 for the first month.*) At the same time, News Corp also forbad the UK’s Audit Bureau of Circulations from reporting site traffic*, so that no meaningful measure of the paywall’s effect was available.

That situation has now been partially reversed, with News reporting some of its own numbers: they claim 105,000 total transactions for digital content between July and October.* (Several people have wrongly reported this as 105,000 users. The number of users is smaller, as there can be more than one transaction per user.) News Corp notes that about half of those transactions were one-offs, meaning only about 50,000 transactions in those four months were by people with any commitment to the site longer than a single day.

Because that 50K number includes not just web site transactions, but Kindle and iPad sales as well, web subscribers are, at best, in the low tens of thousands. However, we don’t know how small the digital subscriber number is, for two reasons. First, the better the Kindle and iPad numbers are, the worse the web numbers are. Second, News did not report, for example, whether a loyal reader from July to October would count as a single transaction or several consecutive transactions. (If iPad sales are good, and loyal users create multiple transactions, then monthly web subscribers could be under 10,000.)

The other figure News reported is that something like 100,000 print subscribers have requested web access. Combining digital-only and print subscribers, and comparing them with comScore’s pre-paywall estimate of roughly six million unique readers worldwide*, the reduction in total web audience seems to be on the order of 97%. (Note that this reduction can’t be measured by before and after traffic, as the home pages are outside the paywall, so people who refuse to pay still show up as visitors.)

Because the print subscribers outnumber digital-only users, most of the remaining 3% pay nothing for the site. Subscription to the paper is now a better draw for website use than any case News has been able to make for paid access.

Given the paucity of the data, the key question of churn remains unanswerable. After the introductory £1 a month offer, the annualized rate rises from £12 to £104. This will cause additional users to bail out, but we have no way of guessing how many.

As with every aspect of The Times’ paywall, interpretation of these numbers varies widely. There are people arguing that these numbers are good news; Robert Andrews at PaidContent sees hope in the Times now having recurring user revenues.* There are people arguing that they are bad news; Mike Masnick at TechDirt believes those revenues are unlikely to offset new customer acquition costs and the loss of advertising.* What is remarkable though, what seems to need more explaining than News’s strategy itself, is why anyone regards this particular paywall as news at all.

* * *

The “paywall problem” isn’t particularly complex, either in economic or technological terms. General-interest papers struggle to make paywalls work because it’s hard to raise prices in a commodity market. That’s the problem. Everything else is a detail.

The classic description of a commodity market uses milk. If you own the only cow for 50 miles, you can charge usurious rates, because no one can undercut you. If you own only one of a hundred such cows, though, then everyone can undercut you, so you can’t charge such rates. In a competitive environment like that, milk becomes a commodity, something whose price is set by the market as a whole.

Owning a newspaper used to be like owning the only cow, especially for regional papers. Even in urban markets, there was enough segmentation–the business paper, the tabloid, the alternative weekly–and high enough costs to keep competition at bay. No longer.

The internet commodifies the business of newspapers. Any given newspaper competes with a few other newspapers, but any newspaper website compete with all other websites. As Nicholas Carr pointed out during the 2009 pirate kidnapping, Google News found 11,264 different sources for the story, all equally accessible.* The web puts newspapers in competition with radio and TV stations, magazines, and new entrants, both professional and amateur. It is the war of each against all.

None of this is new. The potential disruptive effects of the internet on newspapers have been observable since ClariNet in 1989.* Nor has the business case for paywalls changed. The advantage of paywalls is that they raise revenue from users. The disadvantages are that they reduce readership, increase customer acquistion and retention costs, and eliminate ad revenue from user-forwarded content. In most cases, the disadvantages have outweighed the advantages.

So what’s different about News paywall? Nothing. It’s no different from other pay-for-access plans, whether the NY Times’ TimesSelect* or the Harligen Texas Valley Morning Star.* News Corp has produced no innovation in content, delivery, or payment, and the idea of 90%+ loss of audience was already a rule of thumb over a decade ago. Yet something clearly feels different.

Over the last fifteen years, many newspaper people have assumed continuity with the analog business model, which is to say they assumed that readers could eventually be persuaded or forced pay for digital editions. This in turn suggested that the failure of any given paywall was no evidence of anything other than the need to try again.

What is new about the Times’ paywall–what may in fact make it a watershed–isn’t strategy or implementation. What’s new is that it has launched as people in the news business are re-thinking assumed continuity. It’s new because the people paying attention to it are now willing to regard the results as evidence of something. To the newspaper world, TimesSelect looked like an experiment. The Times and Sunday Times look like a referendum on the future.

* * *

One way to escape a commodity market is to offer something that isn’t a commodity. This has been the preferred advice of people committed to the re-invention of newspapers. It is a truism bordering on drinking game material that anyone advising newspapers will at some point say “All you need to do is offer a product so relevant and valuable the consumer is willing to pay for it!”

This advice is well-meaning. It’s just not much help. The suggestion that newspapers should, in the future, create a digital product users are willing to pay for is merely a restatement of the problem, by way of admission that the current product does not pass that test.

Most of the historical hope for paywalls assumed that through some combination of reader desire and supplier persuasiveness, the current form of the newspaper could survive the digital transition without significant alteration.

Payalls, as actually implemented, have not accomplished this. They don’t expand revenue from the existing audience, they contract the audience to that subset willing to pay. Paywalls do indeed help newspapers escape commodification, but only by ejecting the readers who think of the product as a commodity. This is, invariably, most of them.

* * *

You can see this contraction at the Times and Sunday Times in the reversal of digital to print readers. Before the paywall, the two sites had roughly six times more readers than there were print sales of the paper edition. (6M web vs. 1M print for the Sunday Times* .) Post-paywall, the web audience is less than a sixth of print sales (down to <150K vs. 1M). The paying web audience is less a twentieth of print sales (<50K vs. 1M), and possibly much less.

One way to think of this transition is that online, the Times has stopped being a newspaper, in the sense of a generally available and omnibus account of the news of the day, broadly read in the community. Instead, it is becoming a newsletter, an outlet supported by, and speaking to, a specific and relatively coherent and compact audience. (In this case, the Times is becoming the online newsletter of the Tories, the UK’s conservative political party, read much less widely than its paper counterpart.)

Murdoch and News Corp, committed as they have been to extracting revenues from the paywall, still cannot execute in a way that does not change the nature of the organizations behind the wall. Rather than simply shifting relative subsidy from advertisers to users for an existing product, they are instead re-engineering the Times around the newsletter model, because the paywall creates newsletter economics.

As of July, non-subscribers can no longer read Times stories forwarded by colleagues or friends, nor can they read stories linked to from Facebook or Twitter. As a result, links to Times stories now rarely circulate in those media. If you are going to produce news that can’t be shared outside a particular community, you will want to recruit and retain a community that doesn’t care whether any given piece of news spreads, which means tightly interconnected readerships become the ideal ones. However, tight interconnectedness correlates inversely with audience size, making for a stark choice, rather than offering a way of preserving the status quo.

This re-engineering suggests that paywalls don’t and can’t rescue current organizational forms. They offer instead yet another transformed alternative to it. Even if paywall economics can eventually be made to work with a dramatically reduced audience, this particular referendum on the future (read: the present) of newspapers is likely to mean the end of the belief that there is any non-disruptive way to remain a going concern.


I’ve bundled some replies to various questions in the comments from November 9th here and from November 10th here.

Also, nota bene: One of the problems with the various “Hey you guys, I just had a great idea for saving newpapers!” micropayment comments showing up in my moderation queue is that the proposers often exhibit no understanding that micropayments have a 20-year history of failure.

I will not post comments suggesting that micropayments will save the news industry unless those comment refer to at least some of the theoretical or practical literature on previous attempts to make them work for the news business. Start here: Why Small Payments Won’t Save Publishers

The Collapse of Complex Business Models

April 1, 2010

I gave a talk last year to a group of TV executives gathered for an annual conference. From the Q&A after, it was clear that for them, the question wasn’t whether the internet was going to alter their business, but about the mode and tempo of that alteration. Against that background, though, they were worried about a much more practical matter: When, they asked, would online video generate enough money to cover their current costs?

That kind of question comes up a lot. It’s a tough one to answer, not just because the answer is unlikely to make anybody happy, but because the premise is more important than the question itself.

There are two essential bits of background here. The first is that most TV is made by for-profit companies, and there are two ways to generate a profit: raise revenues above expenses, or cut expenses below revenues. The other is that, for many media business, that second option is unreachable.

Here’s why.

* * *

In 1988, Joseph Tainter wrote a chilling book called The Collapse of Complex Societies. Tainter looked at several societies that gradually arrived at a level of remarkable sophistication then suddenly collapsed: the Romans, the Lowlands Maya, the inhabitants of Chaco canyon. Every one of those groups had rich traditions, complex social structures, advanced technology, but despite their sophistication, they collapsed, impoverishing and scattering their citizens and leaving little but future archeological sites as evidence of previous greatness. Tainter asked himself whether there was some explanation common to these sudden dissolutions.

The answer he arrived at was that they hadn’t collapsed despite their cultural sophistication, they’d collapsed because of it. Subject to violent compression, Tainter’s story goes like this: a group of people, through a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex—agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive—each additional bit of complexity more than pays for itself in improved output—but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The ‘and them some’ is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn’t these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn’t because they don’t want to, it’s because they can’t.

In such systems, there is no way to make things a little bit simpler – the whole edifice becomes a huge, interlocking system not readily amenable to change. Tainter doesn’t regard the sudden decoherence of these societies as either a tragedy or a mistake—”[U]nder a situation of declining marginal returns collapse may be the most appropriate response”, to use his pitiless phrase. Furthermore, even when moderate adjustments could be made, they tend to be resisted, because any simplification discomfits elites.

When the value of complexity turns negative, a society plagued by an inability to react remains as complex as ever, right up to the moment where it becomes suddenly and dramatically simpler, which is to say right up to the moment of collapse. Collapse is simply the last remaining method of simplification.

* * *

In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT’s famous “five 9′s” reliability (services that work 99.999% of the time) would be valuable, but they couldn’t figure out how $20 a month, then the going rate, could cover the costs for good web hosting, much less leave a profit.

I started describing the web hosting I’d used, including the process of developing web sites locally, uploading them to the server, and then checking to see if anything had broken.

“But if you don’t have a staging server, you’d be changing things on the live site!” They explained this to me in the tone you’d use to explain to a small child why you don’t want to drink bleach. “Oh yeah, it was horrible”, I said. “Sometimes the servers would crash, and we’d just have to re-boot and start from scratch.” There was a long silence on the other end, the silence peculiar to conference calls when an entire group stops to think.

The ATT guys had correctly understood that the income from $20-a-month customers wouldn’t pay for good web hosting. What they hadn’t understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn’t very good.

This, for the ATT guys, wasn’t depressing so much as confusing. We finished up the call, and it was polite enough, but it was perfectly clear that there wasn’t going to be a consulting gig out of it, because it wasn’t a market they could get into, not because they didn’t want to, but because they couldn’t.

It would be easy to regard this as short-sighted on their part, but that ignores the realities of culture. For a century, ATT’s culture had prized—insisted on—quality of service; they ran their own power grid to keep the dial-tone humming during blackouts. ATT, like most organizations, could not be good at the thing it was good at and good at the opposite thing at the same time. The web hosting business, because it followed the “Simplicity first, quality later” model, didn’t just present a new market, it required new cultural imperatives.

* * *

Dr. Amy Smith is a professor in the Department of Mechanical Engineering at MIT, where she runs the Development Lab, or D-Lab, a lab organized around simple and cheap engineering solutions for the developing world.

Among the rules of thumb she offers for building in that environment is this: “If you want something to be 10 times cheaper, take out 90% of the materials.” Making media is like that now except, for “materials”, substitute “labor.”

* * *

About 15 years ago, the supply part of media’s supply-and-demand curve went parabolic, with a predictably inverse effect on price. Since then, a battalion of media elites have lined up to declare that exactly the opposite thing will start happening any day now.

To pick a couple of examples more or less at random, last year Barry Diller of IAC said, of content available on the web, “It is not free, and is not going to be,” Steve Brill of Journalism Online said that users “just need to get back into the habit of doing so [paying for content] online”, and Rupert Murdoch of News Corp said “Web users will have to pay for what they watch and use.”

Diller, Brill, and Murdoch seem be stating a simple fact—we will have to pay them—but this fact is not in fact a fact. Instead, it is a choice, one its proponents often decline to spell out in full, because, spelled out in full, it would read something like this:

“Web users will have to pay for what they watch and use, or else we will have to stop making content in the costly and complex way we have grown accustomed to making it. And we don’t know how to do that.”

* * *

One of the interesting questions about Tainter’s thesis is whether markets and democracy, the core mechanisms of the modern world, will let us avoid complexity-driven collapse, by keeping any one group of elites from seizing unbroken control. This is, as Tainter notes in his book, an open question. There is, however, one element of complex society into which neither markets nor democracy reach—bureaucracy.

Bureaucracies temporarily suspend the Second Law of Thermodynamics. In a bureaucracy, it’s easier to make a process more complex than to make it simpler, and easier to create a new burden than kill an old one.

In spring of 2007, the web video comedy In the Motherhood made the move to TV. In the Motherhood started online as a series of short videos, with viewers contributing funny stories from their own lives and voting on their favorites. This tactic generated good ideas at low cost as well as endearing the show to its viewers; the show’s tag line was “By Moms, For Moms, About Moms.”

The move to TV was an affirmation of this technique; when ABC launched the public forum for the new TV version, they told users their input “might just become inspiration for a story by the writers.”

Or it might not. Once the show moved to television, the Writers Guild of America got involved. They were OK with For and About Moms, but By Moms violated Guild rules. The producers tried to negotiate, to no avail, so the idea of audience engagement was canned (as was In the Motherhood itself some months later, after failing to engage viewers as the web version had).

The critical fact about this negotiation wasn’t about the mothers, or their stories, or how those stories might be used. The critical fact was that the negotiation took place in the grid of the television industry, between entities incorporated around a 20th century business logic, and entirely within invented constraints. At no point did the negotiation about audience involvement hinge on the question “Would this be an interesting thing to try?”

* * *

Here is the answer to that question from the TV executives.

In the future, at least some methods of producing video for the web will become as complex, with as many details to attend to, as television has today, and people will doubtless make pots of money on those forms of production. It’s tempting, at least for the people benefitting from the old complexity, to imagine that if things used to be complex, and they’re going to be complex, then everything can just stay complex in the meantime. That’s not how it works, however.

The most watched minute of video made in the last five years shows baby Charlie biting his brother’s finger. (Twice!) That minute has been watched by more people than the viewership of American Idol, Dancing With The Stars, and the Superbowl combined. (174 million views and counting.)

Some video still has to be complex to be valuable, but the logic of the old media ecoystem, where video had to be complex simply to be video, is broken. Expensive bits of video made in complex ways now compete with cheap bits made in simple ways. “Charlie Bit My Finger” was made by amateurs, in one take, with a lousy camera. No professionals were involved in selecting or editing or distributing it. Not one dime changed hands anywhere between creator, host, and viewers. A world where that is the kind of thing that just happens from time to time is a world where complexity is neither an absolute requirement nor an automatic advantage.

When ecosystems change and inflexible institutions collapse, their members disperse, abandoning old beliefs, trying new things, making their living in different ways than they used to. It’s easy to see the ways in which collapse to simplicity wrecks the glories of old. But there is one compensating advantage for the people who escape the old system: when the ecosystem stops rewarding complexity, it is the people who figure out how to work simply in the present, rather than the people who mastered the complexities of the past, who get to say what happens in the future.

Secretary Clinton’s Internet Freedom Speech, Abridged

January 23, 2010

[Ed. note: I attended Secretary Clinton's speech on internet freedom on Thursday the 21st, which I thought was a good combination of principle, policy, and illustrative stories. Talking to people afterwards, the commonest question was "What did the Secretary commit the State Department to?" The text below is my attempt to answer that question.

I don't have any inside information about the particulars of the State department's plans; the text below is simply an abridged version of the speech, from which I removed everything except statements you could judge future actions of the State Department on. Stripped of it's context (and with my apologies to the speech writers), my read of the speech is that the success or failure of our internet freedom policy will come down to our ability to live up to the principles outlined below. -clay]


[Topic Sentence]

We stand for a single internet where all of humanity has equal access to knowledge and ideas.

[Internet freedom as an aspect of human rights]

We need to synchronize our technological progress with our principles. In accepting the Nobel Prize, President Obama spoke about the need to build a world in which peace rests on the inherent rights and dignities of every individual. And in my speech on human rights at Georgetown a few days later, I talked about how we must find ways to make human rights a reality. Today, we find an urgent need to protect these freedoms on the digital frontiers of the 21st century.

[Freedom of expression]

But the internet is a network that magnifies the power and potential of all others. And that’s why we believe it’s critical that its users are assured certain basic freedoms. Freedom of expression is first among them. This freedom is no longer defined solely by whether citizens can go into the town square and criticize their government without fear of retribution. Blogs, emails, social networks, and text messages have opened up new forums for exchanging ideas, and created new targets for censorship.

[Exceptions for incitement to violence; disapproval of hate speech]

Now, all societies recognize that free expression has its limits. We do not tolerate those who incite others to violence, such as the agents of al-Qaida who are, at this moment, using the internet to promote the mass murder of innocent people across the world. And hate speech that targets individuals on the basis of their race, religion, ethnicity, gender, or sexual orientation is reprehensible.

[Acknowledgment of hard cases, which are nevertheless subordinate to peaceful political uses]

And we must also grapple with the issue of anonymous speech. Those who use the internet to recruit terrorists or distribute stolen intellectual property cannot divorce their online actions from their real world identities. But these challenges must not become an excuse for governments to systematically violate the rights and privacy of those who use the internet for peaceful political purposes.

[Freedom of worship]

But the freedom of worship also speaks to the universal right to come together with those who share your values and vision for humanity. In our history, those gatherings often took place in churches, synagogues, mosques and temples. Today, they may also take place on line.

Now, just as these technologies must not be used to punish peaceful political speech, they must also not be used to persecute or silence religious minorities.

We must work to advance the freedom of worship online just as we do in other areas of life.

[International coordination on cybersecurity]

We need more tools to help law enforcement agencies cooperate across jurisdictions when criminal hackers and organized crime syndicates attack networks for financial gain. The same is true when social ills such as child pornography and the exploitation of trafficked women and girls online is there for the world to see and for those who exploit these people to make a profit. We applaud efforts such as the Council on Europe’s Convention on Cybercrime that facilitate international cooperation in prosecuting such offenses. And we wish to redouble our efforts.

Countries or individuals that engage in cyber attacks should face consequences and international condemnation.

[Freedom to connect, likened to freedom of assembly]

The final freedom, one that was probably inherent in what both President and Mrs. Roosevelt thought about and wrote about all those years ago, is one that flows from the four I’ve already mentioned: the freedom to connect – the idea that governments should not prevent people from connecting to the internet, to websites, or to each other.

The freedom to connect is like the freedom of assembly, only in cyberspace.

[USG support for the broad goals]

The United States is committed to devoting the diplomatic, economic, and technological resources necessary to advance these freedoms.

[Support for dissidents; return to theme of human rights]

And I’m proud that the State Department is already working in more than 40 countries to help individuals silenced by oppressive governments. We are making this issue a priority at the United Nations as well, and we’re including internet freedom as a component in the first resolution we introduced after returning to the United Nations Human Rights Council.

[USG funding research into freedom-supporting tools]

We are also supporting the development of new tools that enable citizens to exercise their rights of free expression by circumventing politically motivated censorship. We are providing funds to groups around the world to make sure that those tools get to the people who need them in local languages, and with the training they need to access the internet safely.

[USG support for the broad goals II]

Both the American people and nations that censor the internet should understand that our government is committed to helping promote internet freedom.

[USG funding research into freedom-supporting tools II]

That’s why today I’m announcing that over the next year, we will work with partners in industry, academia, and nongovernmental organizations to establish a standing effort that will harness the power of connection technologies and apply them to our diplomatic goals. By relying on mobile phones, mapping applications, and other new tools, we can empower citizens and leverage our traditional diplomacy. We can address deficiencies in the current market for innovation.

And the State Department will be launching an innovation competition to give this work an immediate boost. We’ll be asking Americans to send us their best ideas for applications and technologies that help break down language barriers, overcome illiteracy, connect people to the services and information they need.

[Pro-freedom agenda in the market]

Increasingly, U.S. companies are making the issue of internet and information freedom a greater consideration in their business decisions. I hope that their competitors and foreign governments will pay close attention to this trend.

And censorship should not be in any way accepted by any company from anywhere. And in America, American companies need to make a principled stand. This needs to be part of our national brand.

[Support for Global Internet Freedom Task Force]

Now, we are reinvigorating the Global Internet Freedom Task Force as a forum for addressing threats to internet freedom around the world, and we are urging U.S. media companies to take a proactive role in challenging foreign governments’ demands for censorship and surveillance. The private sector has a shared responsibility to help safeguard free expression.

[February 2010 meeting with network services firms]

As part of our commitment to support responsible private sector engagement on information freedom, the State Department will be convening a high-level meeting next month co-chaired by Under Secretaries Robert Hormats and Maria Otero to bring together firms that provide network services for talks about internet freedom, because we want to have a partnership in addressing this 21st century challenge.