Clay Shirky

Writing in his blog on the New York Times yesterday, David Pogue, one of the Times’ tech columnists, advises toning down the alarmist rhetoric over SOPA, suggesting that opponents of the bill (and its Senate cousin PIPA) should Put Down the Pitchforks. He takes particular issue with people who have criticized SOPA without actually understanding the text of the bill. Then, after this preamble, Pogue proceeds to offer an explanation of SOPA that makes it clear that he does not understand the text of the bill.

Here’s his description of what’s at stake:

If the entertainment industry’s legal arm gets out of control, [opponents] say, they could deem almost anything to be a piracy site. YouTube could be one, because lots of videos include bits of TV shows and copyrighted music. Facebook could be one, because people often link to copyrighted videos and songs. Google and Bing would be responsible for removing every link to a questionable Web site. Just a gigantic headache.

That’s Pogue’s perspective: Letting Hollywood decide whether any given site with user contributions facilitates piracy would amount to nothing more than “a gigantic headache.” (Me, I’d have gone with “a violation of the First Amendment.”) To come to a conclusion like this, you’d have to believe that traditional media companies are committed to balancing their desire for control with a respect for citizen rights, and indeed, Pogue does seem to believe this (hence the observation that bad things would happen only if the entertainment industry’s legal arm gets out of control.)

If their legal arm gets out of control? This is an industry that demands payment from summer camps if the kids sing Happy Birthday or God Bless America, an industry that issues takedown notices for a 29-second home movie of a toddler dancing to Prince. Traditional American media firms are implacably opposed to any increase in citizens’ ability to create, copy, save, alter, or share media on our own. They fought against cassette audio tapes, and photocopiers. They swore the VCR would destroy Hollywood. They tried to kill Tivo. They tried to kill MiniDisc. They tried to kill player pianos. They do this whenever a technology increases user freedom over media. Every time. Every single time.

And they don’t just want control — they want it at low cost, and high speed. Pogue talks about the bills’ allowing the Government to sue. What he doesn’t mention is that the bills were also written to allow “market based” system allowing media firms to get injunctions against sites they don’t like, or that they were written so that firms who host user conversations would have incentives to censor their own users in advance, rather than waiting for notification from a copyright holder, as happens now.

I know David Pogue, and he’s a smart guy. I don’t think he’s intentionally trying to obscure the way the bill imagines letting media firms escape due process and impose “market-based” censorship. I think he simply cannot imagine that the bills are as bad as they actually are.

This is a general problem — there is a reasonable conversation to be had about sites set up for large, commercial operations that are designed to violate copyright. And because there’s a reasonable conversation to be had, Pogue (and many others) simply imagine that the core of SOPA must therefore be reasonable. Surely Hollywood wouldn’t try to suspend due process, would they? Or create a parallel enforcement system? Or take away citizen recourse if they were unfairly silenced? They wouldn’t imagine the possibility of a longer jail term for streaming a Michael Jackson video than Jackson’s own doctor got for killing actual Michael Jackson? Would they?

Hollywood wants to take the law into their own hands — they had our representatives add a vigilante clause, for God’s sake, to protect overzealous censors from legal challenge by users — and like a Scooby Doo™ episode, they would have gotten away with it too, if it hadn’t been for us meddlesome kids.

Chris Dodd, lobbyist-in-chief for the MPAA, who is watching the thick end of a hundred million bucks of paid-for legislation swirl around the drain, has been reduced to bizarrely indirect defensiveness, touting the First Amendment credentials of the bill’s co-sponsors, as if that meant these bills must therefore be clean as well. Yet the very first substantive section of SOPA, Section 2.a.1., gives the game away, by being a little too touchy about its constitutional implications: “Nothing in this Act shall be construed to impose a prior restraint on free speech.” Got that? This bill is not about prior restraint. Totally not! What would make you even think such a thing!?

And arguments like Pogue’s are dangerous not because they are pro-SOPA — Pogue himself is glad it is in trouble — but because they obscure the core historical fact: The American media industry tries to stifle user freedom. Every time. Every single time.

We should delight in the stand we’ve taken in favor of things like, say, notifications, and trials, and proof before censoring someone, but we should get ready to do it again next year, and the year after that. The risk now is not that SOPA will pass. The risk is that we’ll think we’ve won. We haven’t; they’ll be back. Get ready to have this fight again.

This may be the year where newspapers finally drop the idea of treating all news as a product, and all readers as customers.

One early sign of this shift was the 2010 launch of paywalls for the London Times and Sunday Times. These involved no new strategy; however, the newspaper world was finally willing to regard them as real test of whether general-interest papers could induce a critical mass of readers to pay. (Nope.) Then, in March, the New York Times introduced a charge for readers who crossed a certain threshold of article views (a pattern copied from the financial press, and especially the Financial Times) which is generating substantial revenue. Finally, and most recently, were a pair of announcements last month: The Chicago Sun-Times was adopting a new threshold charge, and the Minneapolis Star-Tribune said that their existing one was also working well. Taken together, these events are a blow to the idea that online news can be treated as a simple product for sale, as the physical newspaper was.

For some time now, newspaper people have been insisting, sometimes angrily, that we readers will soon have to pay for content (an assertion that had already appeared, in just that form, by 1996.) During that same period, freely available content grew ten-thousand-fold, while buyers didn’t. In fact, as Paul Graham has pointed out, “Consumers never really were paying for content, and publishers weren’t really selling it either…Almost every form of publishing has been organized as if the medium was what they were selling, and the content was irrelevant.”

Commercial radio is ad-supported because no one could figure out a way to restrict access to radio waves; cable TV collects revenues because someone figured out a way to restrict access to co-axial cables. The logic of the internet is that everyone pays for the infrastructure, then everyone gets to use it. This is obviously incompatible with print economics, but oddly, the industry’s faith in ‘every reader a customer’ has been largely unshaken by newspapers’ own lived experience of the move to the web.

A printed paper was a bundle. A reader who wanted only sports and stock tables bought the same paper as a reader who wanted local and national politics, or recipes and horoscopes. Online, though, that bundle is torn apart, every day, by users who forward each other individual URLs, without regard to front pages or named sections or intended navigation. This unbundling leads to the odd math of web readership — if you rank readers by pages viewed in a month, the largest group by far, between a third and half of them, will visit only a single page. A smaller group will read two pages in a month, a still smaller group will read three, and so on, up to the most active reader, in a group by herself, who will read dozens of pages a day, hundreds in a month.

Against this hugely variable audience behavior, a paywall was all-or-nothing: “If you won’t give us any money, we won’t show you any ads!” Offered this all-or-nothing choice, most readers opted for ‘nothing’; the day they launched their paywall, the Times of London shrank its digital audience from a large multiple of its print circulation to a small fraction of it. This isn’t a problem with general-interest paywalls — it is the problem, widely understood before the turn of the century, and one to which there has never been a convincing answer. The easy part of treating digital news as a product is getting money from 2% of your audience. The hard part is losing 98% of your advertising base.

* * *

To understand newspapers’ 15-year attachment to paywalls, you have to understand “Everyone must pay!” not just as an economic assertion, but as a cultural one. Though the journalists all knew readership would plummet if their paper dropped imported content like Dear Abby or the funny pages, they never really had to know just how few people were reading about the City Council or the water main break. Part of the appeal of paywalls, even in the face of their economic ineffectiveness, was preserving this sense that a coupon-clipper and a news junkie were both just customers, people whose motivations the paper could serve in general, without having to understand in particular.

The article threshold has often been discussed as if it was simply a new method of getting readers to pay, to which the reply has to be “Yes, except for most of them.” Calling article thresholds a “leaky” or “porous” paywall understates the enormity of the change; the metaphor of a leak suggests a mostly intact container that lets out a minority of its contents, but a paper that shares even two pages a month frees a majority of users from any fee at all. By the time the threshold is at 20 pages (a number fast becoming customary) a paper has given up on even trying to charge between 85% and 95% of its readers, and it will only convince a minority of that minority to pay.

Newspapers have two principal sources of revenue, readers and advertisers, and they can operate at mass or niche scale for each of those groups. A metro-area daily paper is a mass product for customers (many readers buy the paper) and for advertisers (many readers see their ads.) Newsletters and small-circulation magazines, by contrast, serve niche readers, and therefore niche advertisers — Fire Chief, Mother Earth News. (Some newsletters get by with no advertising at all, as with Cooks’ Illustrated, where part of what the user pays for is freedom from ads, or rather freedom from a publisher beholden to advertisers.)

Paywalls were an attempt to preserve the old mass+mass model after a transition to digital distribution. With so few readers willing to pay, and therefore so few readers to advertise to, paywalls instead turned newspapers into a niche+niche business. What the article threshold creates is an odd hybrid — a mass market for advertising, but a niche market for users. As David Cohn has pointed out, this is the commercial equivalent of the National Public Radio model, where sponsors reach all listeners, but direct suport only comes from donors. (Lest NPR seem like small ball, it’s worth noting that the Times ‘ has convinced something like one out of every hundred of its online readers to pay, while NPR affiliates’ success rate is something like one in twelve. Newspapers with thresholds now aspire to NPR’s persuasiveness.) Paywalls encourage a paper to focus on the value of their content. Thresholds encourage them to focus on the value of their users.

* * *

Threshold charges subject the logic of the print bundle — a bit of everything for everybody, slathered with ads — to two new questions: What do our most committed users want? And what will turn our most frequent readers into committed users? Here are some things that won’t: More ads. More gossip. More syndicated copy. This is new territory for mainstream papers, who have always had head count rather than engagement as their principal business metric.

Celebrities behaving badly always drive page-views through the roof, but those readers will be anything but committed. Meanwhile, the people who hit the threshold and then hand over money are, almost by definition, people who regard the paper not just as an occasional source of interesting articles, but as an essential institution, one whose continued existence is vital no matter what today’s offerings are.

In discussing why the most loyal subset of readers would pay for access to the Times, Felix Salmon described some of the motivations reported by users: “I like the product, understand the incentives involved, and want its production to continue” and “I feel that maintaining a quality NYT is immensely important to the country as a whole.” Now, and presumably from now on, the readers that matter most are disproportionately likely to score high on the God Forbid index (as in “God forbid the Sun-Times not be around to keep an eye on the politicians!”)

The people who feel this way have always been a minority of the readership, a fact obscured by print bundles, but made painfully visible by paywalls. When a paper abandons the standard paywall strategy, it gives up on selling news as a simple transaction. Instead, it must also appeal to its readers’ non-financial and non-transactional motivations: loyalty, gratitude, dedication to the mission, a sense of identification with the paper, an urge to preserve it as an institution rather than a business.

* * *

Thresholds are now mostly being tried at big-city papers — New York, Chicago, Minneapolis. Most papers, however, are not the Minneapolis Star-Tribune. Most papers are the Springfield Reporter, papers with a circulation 20,000 or less, and mostly made up of content bought from the Associated Press and United Media. These papers may not do well on the God Forbid index, because they produce so little original content, and they may not find thresholds financially viable, because the most engaged hundredth of their audience will number in the dozens, not the thousands.

On the other hand, local reporting is almost the only form of content for which the local paper is the sole source, so it’s also possible to imagine a virtuous circle for at least some small papers, where a civically-minded core of citizens step in to fund the paper in return for an increase in local coverage, both of politics and community matters. (It’s hard to overstate how vital community coverage is for small-town papers, which have typically been as much village well as town crier.)

It’s too early to know what behaviors the newly core users will reward or demand from their papers. They may start asking to see fewer or less intrusive ads than non-paying readers do. They may reward papers that make their comments section more conversational (as the Times has just done.) The most dramatic change, though, is that the paying users are almost certain to be more political engaged than the median reader.

There has never been a mass market for good journalism in this country. What there used to be was a mass market for print ads, coupled with a mass market for a physical bundle of entertainment, opinion, and information; these were tied to an institutional agreement to subsidize a modicum of real journalism. In that mass market, the opinions of the politically engaged readers didn’t matter much, outnumbered as they were by people checking their horoscopes. This suited advertisers fine; they have always preferred a centrist and distanced political outlook, the better not to alienate potential customers. When the politically engaged readers are also the only paying readers, however, their opinion will come to matter more, and in ways that will sometimes contradict the advertisers’ desires for anodyne coverage.

It will take time for the economic weight of those users to affect the organizational form of the paper, but slowly slowly, form follows funding. For the moment at least, the most promising experiment in user support means forgoing mass in favor of passion; this may be the year where we see how papers figure out how to reward the people most committed to their long-term survival.

Dean Starkman has written a lengthy piece in the Columbia Journalism Review, assessing the writings of a group of us he calls the “Future of News” movement. That essay, Confidence Game, focusses principally on Jay Rosen and me, both of NYU’s Carter Institute, and Jeff Jarvis of CUNY, though noting some similarity of vision with Emily Bell of Columbia, Dan Gillmor of Arizona, and John Paton, publisher of the Journal-Register Company. (Unmentioned fellow travelers include, mutatis mutandis, Steve Yelvington, Chris Anderson, Amanda Michel, Steve Buttry, Jonathan Stray, and Alan Mutter.)

Starkman doesn’t just criticize us (though he does that, at length.) He also puts forward a Burkean defense of institutional tradition as a store of embedded wisdom, arguing for the continued relevance of existing news organizations, especially newspapers, in something very close to their current form.

He jokingly calls his vision the “Neo-Institutional Hub-and-Spoke Model.” His description of what’s needed, though — “rebuilding or shoring up institutions” — makes it clear he doesn’t need the “Neo-” bit. He is talking about somehow saving the familiar institutions, not inventing new ones, a strategy that has long passed for Plan A in the conversation about what the internet changes about the news business.

He’s not even wild about the familiar institutions altering themselves too radically to accomodate those changes. Paton, who is trying to save local news reporting, is derided for being a “FON practitioner” (shades of the 5th column), and The Guardian, a storied paper since back in the day, gets ten with the cane for going ‘digital first‘. When echt newspaper guys like Paton of the Journal-Register and Alan Rusbridger of The Guardian are seen as heretics, you know we’re talking about that old-time religion.

Rosen and Jarvis and Bell and I disagree plenty, but one belief we have in common is that the way newspapers used to be organized and funded is a bad fit for the current environment, and getting worse. More than any individual sentiment we may have expressed, our public loss of faith in the institutional logic of Plan A seems to be what has most aroused Starkman’s ire.

* * *

Institutions reduce the choices available to their members. (This is Ronald Coase’s famous argument about transaction costs.) This reduction allows better focus on the remaining choices they face.

The editors meet every afternoon to discuss the front page. They have to decide whether to put the Mayor’s gaffe there or in Metro, whether to run the picture of the accused murderer or the kids running in the fountain, whether to put the Biker Grandma story above or below the fold. Here are some choices they don’t have to make at that meeting: Whether to have headlines. Whether to be a tabloid or a broadsheet. Whether to replace the entire front page with a single ad. Whether to drop the whole news-coverage thing and start selling ice cream.

Every such meeting, in other words, involves a thousand choices, but not a billion, because most of the big choices have already been made. These frozen choices are what gives institutions their vitality — they are in fact what make them institutions. Freed of the twin dangers of navel-gazing and random walks, an institution can concentrate its efforts on some persistent, medium-sized, and tractable problem, working at a scale and longevity unavailable to its individual participants.

Institutions also reduce the choices a society has to make. In the second half of the 20th century, “the news” was whatever was in the newspaper on the morning, or network TV at night. Advertisers knew where to reach shoppers. Politicians knew who to they had to talk to to get their message out (sometimes voluntarily, sometimes not.) Readers understood an Letters page as the obvious way of getting wider circulation for their views.

That dual reduction of choices masks an essential asymmetry, though. Institutions are designed to reduce they choices for their members, but they only happen to reduce the choices in society. A publisher may want reporters at their desks at 10 am, and to be the main source of breaking news for the paper’s readers. The former desire is under the publisher’s control; the latter not.

Now this seems easy enough to describe, but people inside institutions tend to confuse the two, to believe their institution is in control of both their daily routine and their destiny. And of course, the longer any given institution maintains a stable role, the less that role seems like an accident and the more it seems like a robust feature of reality.

* * *

Starkman and I could, I think, agree on this story so far, and indeed, most of the time institutional vitality is a safe bet. The ability of institutions to adapt slowly while preserving continuity of mission and process is exactly what lets them last longer than a single leader or lifespan. When change in the outside world outstrips an institution’s adaptive capabilities, though, the ability to defend the internal organization from outside pressures can become a liability. Stability can tun into rigidity and even institutional blindness.

Newspapers were stable for a really long time. The 1830s saw the rise of the penny press, when advertisers generated enough subsidy for the cover price to be reduced from six cents to one, with an attendant explosion in readership. Improbably enough, that model survived for most of two centuries with little more than tweaks and updates, right up to the sudden increase in freedoms occasioned by the internet, freedoms enjoyed by both advertisers and readers. The erosion of geographic sinecure made every story available everywhere. Casual aggregation from multiple sources delighted readers but eroded loyalty. Competition turned paywalls into cul-de-sacs instead of toll roads. Upstart web services (Yahoo and iVillage, Slate and Salon) first set then compressed a market price for ads that the newspapers never had a chance to alter. Users forwarding stories to one another increased audience size, but destroyed scarcity.

Despite these challenges to newspapers, Starkman believes that we can and must “…find ways to preserve and transfer their most important attributes to a digital era, even as we push them to adapt to new financial, technological, and cultural realities.” I don’t believe we must do this, because I don’t believe we can do this. That, I think, is the core diference between our views.

There are thousands of papers in the US, all facing simultaneous pressures on their revenues, their organizational form, the loyalty of their readership, the ferocity of the competition, and even their sense of self. If, seeing those challenges, we FONeros still thought it was possible to preserve most papers in something like their current form, we’d adopt Plan A too. You’d only pursue something like Plan B if you’d given up on broad institutional preservation in the first place.

* * *

To my eye, Starkman stacks the deck when comparing Plans A and B. He lists only three examples of successful “FON” journalism (US Attorney firings, “macacca,” and “bittergate“), but his recounting of the glories of print go back to Ida Tarbell. Tarbell was indeed a terrific reporter, but her byline has been somewhat scarce of late, given she’s been dead 70 years. Comparing a 5 year stretch of recent experiments with the greatest hits of newspapering since the McKinley administration may rally the home team, but it doesn’t make for a particularly informative comparison.

Like a Yeats of the newspaper world, Starkman yearns for the restoration of a culture considerably purer than the actual newspaper business has ever been. Reading Confidence Game, you’d never know that most papers are not like the NY Times, that most of what appears in their pages is syndicated, that sports is often better represented on the masthead than hard news. You’d never know that more American papers printed today will include a horoscope than international news. You’d never know that newspapers are institutions where grown men and women are assigned to write stories about dogs catching frisbees.

Saying newspapers will provide a stable home for reporters, just as soon as we figure out how to make newspapers stable, is like saying that if we had some ham, we could have a ham sandwich, if we had some bread. We need to support the people who cover hard news, but when you see a metro daily for a town of 100,000 that employs only six such reporters (just 10% of the masthead, much less total staff), saving the entire edifice just to support that handful looks a lot harder than just finding new ways to support them directly.

This view is, as Starkman says, anti-institutional, starting as it does from the premise that the outside world is changing faster than most newspapers’ adaptive capabilities. They have responded to 20 consecutive quarters ad revenue decline with the evisceration of international desks and newsroom staff. More is on the way. No medium has ever survived the indifference of 25 year olds.

* * *

We have long since entered a period of hybridization — William Bastone quitting The Village Voice to found Smoking Gun and Nate Silver selling fivethirtyeight.com to the NY Times, Andy Carvin and Anjali Mullany pioneering new forms of live coverage from inside traditional organizations, Amanda Michel’s career implementing citizen journalism at Huffington Post, then at ProPublica, then at The Guardian, these are concrete demonstrations that the old dichotomies of traditional vs. new media, professionals vs. amateurs, incumbents vs. insurgents, have long since stopped being real, hard choices, and have instead become points on an increasingly traversable spectrum.

All of this seems to offer the grandmotherly option between Starkman and the FON crew — “You’re both right, dear. We need institutions and we need experiments.” Even given this hybridization, though, our views diverge: Plan A assumes that experiments should be spokes to the newspapers’ hub, their continued role as the clear center of public interest journalism assured, and on the terms previously negotiated.

Plan B follows Jonathan Stray’s observations about the digital public sphere: in a world where Wikipedia is a more popular source of information than any newspaper, maybe we won’t have a clear center anymore. Maybe we’ll just have lots of overlapping, partial, competitive, cooperative attempts to arm the public to deal with the world we live in.

Some of the experiments going on today, small and tentative as they are, will eventually harden into institutional form, and that development will be as surprising as the penny press subsidizing journalism for seven generations. The old landscape had institutions and so will the new one, but this doesn’t imply continuity. We still have companies called Western Union and ATT, but as the communications landscape changed, they have become almost unrecognizably different from their former selves. Likewise, as the presses fall silent over the next ten years, even papers that survive will see their internal organization and their place in the ecosystem altered beyond our ability to predict.

If you believed, as Starkman clearly does, that this view is not just incorrect but odious, that the current form of the newspaper remains a good general fit for public interest journalism, merely going through a rough patch, then you’d be eager to dial down the ‘try anything’ ethic in favor of the hard, grinding work of rebuilding and shoring up the institutions that have served us so well these last several decades.

But if you believe, as I do, that many of those institutions are so mismatched to the task at hand that most of them face a choice, at best, between radical restructure and outright collapse, well, in that case, you’d probably find the smartest 25 year olds you know, and try to convince them that now would be a pretty good time to start working on Plan B.

The business environment for newspapers continues to be grim. Pew recently reported that advertising revenue rebounded in 2010 for all forms of media, except newspapers.* This might just be a matter of transitioning from print to digital revenues but for the fact that the market values a print reader far more than a digital one. The more or less official label for this problem is “analog dollars to digital dimes”; because of the enormous difference in assumed value per reader, lost value from print is not made up for by gains in digital readership.

The ‘analog dollars to digital dimes’ problem doesn’t actually seem to be a problem. It seems to be a feature of reality. Digital revenue per head is not replacing lost print revenue and, barring some astonishment in the advertising market, it never will. There is no supply-side scarcity to boost margins; power over aggregation has moved from producer to consumer; advertisers prefer selling cameras and shoes on purpose-built sites; readers feel the same about finding jobs and dates; and there are few commercial or geographic restraints on competition. The market values analog readers more highly than digital ones because the market is right.

Seeing this, several people have started looking for ways to exit that market. One of the most widely circulated of these ideas is David Swenson and Michael Schmidt’s proposal that newspapers be subsidized the way colleges or foundations are.* Steve Coll followed this with a suggestion that commercial papers be allowed to reinvent themselves as non-profits.*

The intuition common to such proposals is that advertising revenues can no longer be relied on to fund serious reporting. (Gadget reviews and celebrity coverage, yes, but not the police beat.) Evidence entered for this thesis includes the decline in print’s commercial fortunes*, the attendant downsizing of newsrooms—already 30% smaller, on average, than a decade ago*—and the outright collapse of papers like the Rocky Mountain News, Albuquerque Post, and Cincinnati Post.*

Proponents of market-supported journalism, among them Jack Shafer*, Alan Mutter*, and Jeff Jarvis*, have replied to these non-profit proposals at length, but their common thesis, distilled to its essence, is “Oh please.” This group points out that there is simply not enough philanthropic money to support current news-gathering organizations, and any suggestion that there someday might be isn’t strategy but fantasy.

Evidence for this thesis includes the gap between supply and demand for philanthropic funding* or from the narrowness of funding sources, as with ProPublica’s getting 80 times as much money from its board as from online donations.*

This argument between Journalism as Philanthropy and Journalism as Capitalism (to borrow Jarvis’s phrase) seems like a dilemma, in the literal sense of the word—a forced choice between two alternative premises. It isn’t really, though, because there is also the grandmotherly option: Both groups could be right. It’s possible that for-profit revenue is shrinking irreversibly and that non-profit funding sources won’t make up for the shortfall. The least we can say about this possibility is that it can’t be discounted on current evidence.

One proposed response is to radically reform newspapers as both organizations and businesses. Here, as one of countless examples, is something from a recent post by Tom Matlack*, the former CFO of The Providence Journal:

[T]here is no longer an uncomfortable ménage à trois between newspapers, readers and advertisers. It’s monogamy at last. Newspapers have to focus all their energy on producing extraordinary journalism—in a form and in substance—which thrills us as consumers. [...] So fear not. Your paper is not dead. You just have to demand, and be willing to pay for, a news product worthy of your affections.

Matlack’s sentiment is clearly heartfelt, and creating a high-quality product for a group of loyal and passionate readers willing to pay for it certainly sounds like an interesting business to get into. It just doesn’t sound like the newspaper business.

Here’s what the newspaper business sounds like: the modestly talented son of the founder can generate double-digit margins based on little more than the happy accident that there are people who like football and buy cars living within 30 miles of his house.

That’s the newspaper business, or at least it was until recently. The average US paper runs more soft than hard news, uses more third-party content than anything created by their own staff, and reaches more people who care about local teams than local zoning. Telling the publishers of those papers to create a digital product so extraordinary that readers will pay full freight is a tacit admission that they do not know how to make such a product today.

Much public worry about newspapers concerns a relative handful of excellent dailies with national or international ambitions. Most papers, however, aren’t like that. The New York Times and the Enid, Oklahoma News and Eagle occupy different parts of the news ecosystem, and they face different stresses and fates, but more papers—many more—exist at the News and Eagle end of the spectrum.*

It’s easy to view the current business climate as a culling of the herd, but newspapers are not barbershops; the closing of a hair-cutting business in Oklahoma wouldn’t much affect people in New Hampshire, but the closing of a paper would, because the nation’s news publications are sewn together in a crazy quilt of shared cost and effort.

The News and Eagle, the Laconia Citizen, the Biloxi Sun-Herald, the Deer Park Tribune, and a thousand more such papers are all but monopoly suppliers of local news, they all train young reporters who go on to work elsewhere, they all employ the stringers who are on the scene when a tornado hits, and they all buy syndicated content from the Associated Press or King Features, who in turn lower costs for some publishers while raising revenue for others. When a paper fires reporters or closes outright, it further weakens that fabric.

Buy a newspaper. Cut it up. Throw away the ads. Sort the remaining stories into piles. Now, describe the editorial logic holding those piles together.

If you’ve picked a general interest paper, this will be hard. I recently learned, from a single day’s paper, that a bombing in Kirkuk killed 27, that Penelope Cruz has only good memories of filming Pirates of the Caribbean while pregnant, that many U.S. business hotels are switching to ‘shower-only’ bathrooms, and that 30-year fixed mortgages fell from 4.63% to 4.61% the week before.

The rationale for creating such a bundle went something like this: “We will print enough content to fill the hole left after we’ve sold the advertising space. We will include content proportional to the amount and intensity of reader interest, modified somewhat by editorial judgment. Overall, the value of the bundle will be more than the sum of its parts.”

Many people who worked for newspapers didn’t think of this as something that happened to work well in certain cities and towns during a particular era, but as something real: “Of course Ratko Mladic’s arrest belongs in the same package as a photo spread on ornamental grasses! How else would you do it?” For all that selling such a bundle was a business, though, people have never actually paid for news. We have, at most, helped pay for the things that paid for the news.

So long as newspapers faced little competition for advertisers or readers, this was a distinction without a difference, but as papers are being sundered by the internet, we can see how tangled the system always was. Outside a relative handful of financial publications, there is no such thing as the news business. There is only the advertising business. The remarkable thing about the newspapers’ piece of that business isn’t that they could reliably generate profits without accomplishing much in the way of innovation—that could just as easily describe the local car dealership. The remarkable thing is that over the last couple of generations, those profits supported the fractional bit of those enterprises that covered the news.

This subsidy relied on cultural logic peculiar to newspapers; publishers were constrained not just by their investors but by their editors (who expected the paper to be ethical in the short term) and by their families (who expected the paper to be viable over the long term). In return, a publisher could extract some of the value of the paper in prestige and sinecure instead of cash.

This system was never ideal—out of the crooked timber of humanity no straight thing was ever made—and long before Craig Newmark and Arianna Huffington began their reign of terror, Gannett and Scripps were pioneering debt-laden balance sheets, highly paid executives, and short-term profit-chasing. But even in their worst days, newspapers supported the minority of journalists reporting actual news, for the minority of citizens who cared. In return, the people who followed sports or celebrities, or clipped recipes and coupons, got to live in a town where the City Council was marginally less likely to be corrupt.

Writing about the Dallas Cowboys in order to take money from Ford and give it to the guy on the City Desk never made much sense, but at least it worked. Online, though, the economic and technological rationale for bundling weakens—no monopoly over local advertising, no daily allotment of space to fill, no one-size-fits-all delivery system. Newspapers, as a sheaf of unrelated content glued together with ads, aren’t just being threatened with unprofitability, but incoherence.

This fall, I’m joining NYU’s journalism program, where, for the first time in a dozen years, I will teach undergraduates. Someone who turns 19 this year will have not one adult memory of the 20th century; for them, the Contract With America, the Monica Lewinsky scandal and the first Gulf War are roughly contemporaneous events, just as, for my 19 year old cohort, the Summer of Love, the Watts’ riots, and Kent State all seemed to have happened in that one busy month we called The 60s. When it comes time to explain the media landscape of the 20th century, I will be teaching my own youth as ancient history.

I could tell these students that when I was growing up, the only news I read was thrown into our front yard by a boy on a bicycle. They might find this interesting, but only in the way I found it interesting that my father had grown up without indoor plumbing. What 19 year olds need to know isn’t how it was in Ye Olden Tymes of 1992; they need to know what we’ve learned about supporting the creation and dissemination of news between then and now. Contemplating what I should tell them, there are only three things I’m sure of: News has to be subsidized, and it has to be cheap, and it has to be free.

News has to be subsidized because society’s truth-tellers can’t be supported by what their work would fetch on the open market. However much the Journalism as Philanthropy crowd gives off that ‘Eat your peas’ vibe, one thing they have exactly right is that markets supply less reporting than democracies demand. Most people don’t care about the news, and most of the people who do don’t care enough to pay for it, but we need the ones who care to have it, even if they care only a little bit, only some of the time. To create more of something than people will pay for requires subsidy.

News has to be cheap because cheap is where the opportunity is right now. For all that the Journalism as Capitalism people can sound like Creflo Dollar mid-sermon, they are right to put their faith in new models for news. If for-profit revenue is shrinking and non-profit funding won’t make up the shortfall, we need much cheaper ways of gathering, understanding, and disseminating news, whether measured in information produced or readers served.

And news has to be free, because it has to spread. The few people who care about the news need to be able to share it with one another and, in times of crisis, to sound the alarm for the rest of us. Newspapers have always felt a tension between their commercial and civic functions, but when a publication drags access to the news itself over to the business side, as with the paywalls at The Times of London or the Tallahassee Democrat, they become Journalism as Luxury. In a future dominated by Journalism as Luxury, elites would still get what they need (a tautology in market economies), but most communities would suffer; imagine Bell, California times a thousand, with no Ruben Vives to go after the the politicians.*

The thing I really want to impress on my students is that the commercial case for news only matters if the profits are used to subsidize reporting the public can see, and that civic virtue may be heart-warming, but it won’t keep the lights on, if the lights cost more than cash on hand. Both sides of the equation have to be solved.

Real news—reporting done for citizens instead of consumers—is a public good. This is true both in the colloquial sense of ‘good for the public’ and in the economic sense of ‘best provisioned for a whole group at once.’

The supplier of last resort for public goods is usually the government, but in the United States, public funding of media has always been politically fraught, outside a few subsidies like reduced postal rates. This leaves us the problem of producing a public good without much in the way of public monies.

Taking cash from advertisers is one way to do this, though a less good one than it used to be. As Jay Rosen points out*, many other ways are possible: NewWest.net gets money from its conference business*; The Guardian from the Scott Trust*. Donations are still another: some organizations have a syndicate of large donors, as with ProPublica * or The American Independent*. Others have many small donors, as with the crowdfunding of Spot.us projects, or the listener donations to NPR.

And, critically, subsidy can be in savings rather than cash. Some of what professionals did in the old model can now be done in combination with amateurs, or crowds, or machines: MAPlight* and PoliGraft* and Sunlight’s Lobbying Tracker* couldn’t track links between money and politics without online databases; the Charlotte News Alliance* and the Tuscon Citizen * rely on local bloggers; the Davis Wiki* and the the Oil Spill Crisis Map* provide structure to user-contributed material; Tackable is betting that the first photographer on the scene will be a citizen with a phone*.

None of the models being tried today are universally adoptable; the most we can say is that each of them happens to work somewhere, at least for the moment. This may seem like weak tea, given the enormity of the current changes, but if our test for any new way of producing news is whether it replaces all the functions of a newspaper, we’ll build things that look like newspapers, and if replicating newspapers online were a good idea, we wouldn’t be in this mess in the first place.

If we adopt the radical view that what seems to be happening is actually happening, then a crisis in reporting isn’t something that might take place in the future. A 30% reduction in newsroom staff, with more to come, means this is the crisis, right now. Any way of creating news that gets cost below income, however odd, is a good way, and any way that doesn’t, however hallowed, is bad.

Having one kind of institution do most of the reporting for most communities in the US seemed like a great idea right up until it seemed like a single point of failure. As that failure spreads, the news ecosystem isn’t just getting more chaotic, we need it to be more chaotic, because we need multiple competing approaches. It isn’t newspapers we should be worrying about, but news, and there are many more ways of getting and reporting the news that we haven’t tried than that we have.

I’m quoted in the New York Times today talking about the difficulty in the State Department’s attempt to portray freedom of speech as a human rights issue, while sidestepping the fact that freedom of speech is a destabilizing force in autocratic governments:

You can’t say ‘All we want is for people to speak their minds, not bring down autocratic regimes’ — they’re the same thing.

“Thing” was a sloppy word to use; what I should have said is that they are the same desire. The State Department can’t say they want people in autocracies to have freedom of speech without admitting that this is an expressly political desire–as are all assertions about human rights, however construed–and that this desire is expressly inimical to autocracy.

The next paragraph of the Times’ piece adds just such a clarification:

He added that the United States could expose itself to charges of hypocrisy if the State Department maintained its support, tacit or otherwise, for autocratic governments running countries like Saudi Arabia or Bahrain while deploying technology that was likely to undermine them.

Freedom of speech is inimical to autocratic control, both as cause and effect. When people are free to publicly contradict assertions made by their government, it makes it harder for those governments to govern unilaterally, and the kind of governments that commit themselves to freedom of speech are usually (perhaps invariably?) democracies.

My friend Zeynep Tufekci of U-Maryland and technosociology.org summed up this idea far better than I did: “The internet isn’t your pony such that you can support free speech and expect it not to have political consequences.”

Of all the images to come out of the North African uprisings, this is the one that consistently makes me tear up.

These are Coptic Christians in Cairo’s Tahrir Square, holding hands around Muslims during Friday prayer, to protect them from surprise attack.

We could do this. We could do this here, in the United States, to keep our Muslim citizens safe. All of us, Christians and Jews and atheists, anyone committed to the First Amendment, to the idea that religious freedom is essential to American society, could join hands to protect Muslims living here from rising prejudice.

The people of the United States are capable of visiting horrors on those we decide aren’t really like “us” enough to count: Blacks owned, women silenced, Jews segregated, gays repressed, Japanese interned, the catalog of American hate and fear runs to many pages.

And now it’s happening again, with political fear-mongering around the Park51 Islamic community center in New York and Rep. Peter King’s recent grandstanding on radicalization in the American Muslim community fueling a ‘guilty until proven innocent’ attitude towards Muslim citizens and organizations. We should certainly worry about murderous homegrown radicals, but that’s because murderous radicals are dangerous, not because they are religious; David Koresh, Timothy McVeigh, and David Brian Stone were all homegrown and all evil, but none of them were Muslim.

I don’t know how bad this kind of prejudice will get, but it will get worse than today. To counter this threat, we have to integrate the Muslim experience into the American experience, just as the Protestants came to shed anti-Catholic prejudice, and the Christians anti-Jewish prejudice. As imperfect as those expansions have been, even today, the United States is a better place for having widened our sense of who constitutes “us,” of having engaged in what Richard Rorty called justice as larger loyalty.

I don’t know how to do this, how to keep people safe from the bigots who want the very fact of Muslim faith or heritage to generate suspicion and oppression. I do know some ways to start. We can respond to prejudice when we hear it; the thing that woke me up was hearing one of my cousins argue that a mosque in lower Manhattan was an affront to American identity, as if Muslim citizens were inherently less American than Christian ones.

We can care about how Muslims are treated, here and abroad. Democratic governments respond to public preferences; bigotry thrives on the silence of those of us who support freedom of religion, but not out loud. (Hence this post.) Democratic governments also respond to voters; it’s a safe bet that the organizations helping register Muslim-American voters in 2008 will do so again in 2012, and they need our help.

We can donate. I’ve given money to the Muslim Public Affairs Council, and to Irshad Manji’s Project Ijtihad (ijtihad meaning, roughly, independent reasoning.) These are of course political choices — Project Ijtihad is explicitly progressive and feminist — and your politics may run more to What Unites Us, an organization set up to combat anti-Muslim prejudice, to the American Muslim Law Enforcement Officers Association, or any number of local or interfaith organizations opposed to religious bigotry.

Perhaps most importantly, we can tell people we’ve donated or participated. It will be better to donate $5 to a group combatting anti-Muslim prejudice and then announce it on Facebook or Twitter, than to give $500 quietly. (I’m using #AfterTahrir.)

As President Bush said at the Islamic Center of Washington

Those who feel like they can intimidate our fellow citizens to take out their anger don’t represent the best of America, they represent the worst of humankind, and they should be ashamed of that kind of behavior.

Bush was right. America’s finest hours have always been integrative and our most shameful ones prejudiced. Now, again, we have to pick, and we have an opportunity to learn from the protesters in Tahrir what political courage looks like.

Over the weekend, Evgeny Morozov asked about my consulting with the Libyan government in 2007. In March of that year, I was invited by Monitor Consulting to come to Boston to speak to a Libyan IT minister about using social software to improve citizen engagement in coastal towns. The idea was that those cities would be more economically successful if local policies related to the tourist trade were designed by the locals themselves. (I talked about that work briefly in this interview [28:30-30:30])

After the meeting in Boston, the imagined project didn’t materialize. I presume this was because the government choked on the inability to get the economic effects of citizen participation without getting the political effects as well, an idea they claimed to want but in fact didn’t.

Morozov also suggests that “anyone who thought they could democratize Libya via Wikis needs to make a public apology.” This is the same caricature he often uses, implying that people like me who think that social tools can improve outcomes actually believe that tools cause those outcomes. What we believed at the time was that Libya’s planned devolution of political power to individual towns was real; what we learned was that it wasn’t. I’m sorry I wasn’t able to help expand representative government in Libya, but I’m not sorry I tried.

I am, in other words, exactly as naive as Morozov accuses me of being. As I’ve said elsewhere, the best reason to believe that social media can aid citizens in their struggle to make government more responsive is that both citizens and governments believe that.

This is a coda to my earlier post, Wikileaks and the Long Haul. It’s an attempt to express a partially formed thought about the Pentagon Papers case and the global media environment.

A bit of potted history first: The Pentagon Papers were a secret history of US involvement in Vietnam, produced by the Department of Defense and leaked in 1971 by Daniel Ellsberg to the NY Times, who published excerpts and analysis from them. The government attempted to prosecute the Times under the Espionage Act; the Times, with Floyd Abrams as their lead attorney, argued the case before the Supreme Court. The Times won, and the decision, New York Times Co. v. United States (403 U.S. 713), established the principle that it was illegal to leak secrets, but not to publish leaks. Justice Hugo Black, writing for the majority, said “Only a free and unrestrained press can effectively expose deception in government.”

I was thinking about Black’s opinion, and particularly his emphasis on an “unrestrained press”, in light of two things I’ve read on Wikileaks. The first, and more recent, is Floyd Abrams tortured attempt to deny that his winning arguments in the Pentagon Papers case set the operative precedent for Wikileaks.

His essay is an attempt to square two things he clearly believes deeply — Wikileaks is bad and must be stopped, and that its actions may not be illegal. He doesn’t want to believe that second thing, but as one of the pre-eminent litigators of 1st Amendment law, he also understands what the precedents say, and glumly comes to this conclusion:

[I]f Mr. Assange were viewed as simply following his deeply held view that the secrets of government should be bared, notwithstanding the consequences, he might escape legal punishment.

Abrams is upset enough about Wikileaks (and Assange) that he can’t bring himself to describe the clear meaning of this conclusion: under the Pentagon Papers precedent, Assange might “escape legal punishment” the ordinary way, by being innocent of having committed a crime.

This in turn brought to mind my NYU colleague Jay Rosen’s observation last summer, during the Afghan War logs leak, that Wikileaks is the first stateless news organization:

Appealing to national traditions of fair play in the conduct of news reporting misunderstands what Wikileaks is about: the release of information without regard for national interest. In media history up to now, the press is free to report on what the powerful wish to keep secret because the laws of a given nation protect it. But Wikileaks is able to report on what the powerful wish to keep secret because the logic of the Internet permits it.

So here’s the half-formed thought: Abrams is wrong that the Pentagon Papers case isn’t the obvious precedent, but he’s right that Wikileaks is so different that the meaning of that case isn’t clear in the present situation.

To draw a distinction between the Pentagon Papers precedent and what Wikileaks means, I’m going to use slightly different language than Rosen’s notion of statelessness. (I think Wikileaks is less stateless than it is multi-homed, allowing it some freedom from traditional ‘single point of censorship’ problems that plague other international media).)

Let me propose, for the sake of argument, two labels for action that spans more than one country: international, and global. International actors are actors rooted in a nation, even when they are able to participate in activities all over the world, while global actors are unrooted; global actors have, as their home environment, the globe.

The closer an activity comes to attaining the condition of pure information, the more global it can be. By way of analogy, the LSD business is more global than the cocaine business, because coca leaves only grow in certain climates, but lysergic acid can be synthesized anywhere. Media is like this as well: The internet is more global than the telephone network, even though both systems can send data between any two points in the world. Similarly, Wikileaks is more global than the BBC or Al Jazeera; those organizations are very large, but they are international, with a home base as rooted in a particular place as a coca farmer is.

The most dramatic of Wikileaks’ breaks with previous journalism is the global nature Rosen identified. The biggest difference between the Pentagon Papers case and Wikileaks is not the legal precedent, but the fact that the Pentagon Papers case was an entirely national affair.

Though some of the most impassioned and vocal participants in the Vietnam war protest movement genuinely cared about the fate of the Vietnamese, the bulk of the participants were animated by a much more proximate goal: ending the draft in the US. The publication of the Pentagon Papers did not seem to have helped end the war, but exposure of the military’s frank internal assessment of the compromised nature of the conflict made it harder to ask middle-class parents to sacrifice their children to that kind of action. (It also helped feed into the collapse of trust in the government and of authority figures generally.)

All this was in the US context, as was every actor involved: Ellsberg, Abrams, the NY Times, the Pentagon, the Supreme Court, the protesting citizens, and so on. No one was out of the reach of the Federal Government, and the effect of the Papers’ publication, in a US new outlet, was a family affair.

Wikileaks has been global from the beginning, and the additional complexity of both jurisdiction and extradition make this particular problem much much more complex than any issues, legal or practical, triggered by the Pentagon Papers. Wikileaks has been operating since 2006, the military has regarded it as a significant threat since at least 2008, and the US Attorney General still has difficulty framing charges he thinks he can win.

For many of our most important social systems, we resolve clashing principles by providing an escape valve, in the form of a set of actors who are less rule-bound than the rest of the system. The most famous and ancient is the jury, a collection of amateurs who can, in the face of clear laws and evidence, simply not return the verdict a judge would have returned.

So with secrecy. Though I am not a lawyer, the Supreme Court’s 1971 ruling seems to say that there is no law-like way to balance the State’s need for secrets with the threats secrets pose to democracies, so it simply said that the 1st Amendment provides immunity to publishers in most circumstances; the presence of publishers as “unrestrained actors” provides one of the many limits on government power that make democracies work.

This immunity sets up publishers as self-regulating checks to government power, albeit in a system that can never be made intellectually coherent — neither total success nor total failure of the government to keep secrets would protect the United States as well as a regime of mostly success with periodic, unpredictable failures.

The Pentagon Papers decision says the government faces a “high bar” to proving that a publisher’s actions are illegal. What’s legal for publishers is thus whatever keeps them under this high bar, but no one knows where this bar is at any given moment, except that it is high enough that no publisher has ever been successfully prosecuted for espionage.

But here’s where Black was wrong, or at least only partly right: the press in the US isn’t unrestrained, just because of this high bar. Instead, the US press is self-restrained. Our curious system of outlawing certain kinds of speech — libel, release of trade secrets, and so on — while also largely forbidding the government from prevention of publication (prior restraint) or creating a climate of fear and doubt for publishers (chilling effects) would seem impossible to balance within a legal system, but that’s because the balance is produced by extra-legal constraints.

A publisher is (or was, in 1971) a commercial, nationally-rooted media firm subject to significant tradeoffs between scale and partisanship. Large publishers had to be deeply embedded in the culture they operated in, and they had to reflect local mainstream views on most matters most of the time, to find and retain both revenue and audience. Fans of game theory will recognize these conditions as those required for an iterated game of the Prisoner’s Dilemma, where the press exhibits self-restraint from short term defection against the US’s interests, in order to benefit from an amicable relationship with the government over the long haul.

This is why the difference between the Times, as an international actor, and Wikileaks, as a global one, matters so much. Wikileaks does not have to play an iterated game of Prisoner’s Dilemma with the US. Not only is Wikileaks not housed in the US, it isn’t housed in any single other nation the US could complain to. They can defect at will. (You can always burn a partner in Prisoner’s Dilemma if they can’t get back at you.) Wikileaks hasn’t defected all the way, of course, releasing only ~2000 of the ~250,000 cables they have; the point is that this restraint is not forced on them by the US.

The legal bargain from 1971 simply does not and cannot produce the outcome it used to. This is one of the things freaking people in the US government out about the long-term change in the media environment — not that the law has changed, but that the world has. Industrial era law, applied to internet-era publishing, might allow for media outlets which exhibit no self-restraint around national sensitivities, because they are run by people without any loyalty to — or, more importantly, need of — national affiliation to do their jobs.

There is, in much of the commentary about Wikileaks online, a kind of echo of Black’s 1971 opinion, a fantasy of unrestrained action. This comes in two flavors: one group believes that since Julian Assange is not a US citizen and not operating on US soil, the US could never charge him with anything. These people seem not to have heard of extradition agreements; the legal system is at least international enough to allow charges to be brought against foreign nationals and have them tried in the US. Whether that will happen to Julian I don’t know, but it’s certainly not out of the realm of the possible.

The other ‘unrestrained action’ group believes that, as Assange is not a US citizen, there is nothing to stop us from simply finding him and having him killed. This group seems not to recognize that political murders are regarded as something of a no-no by our allies, and that such a move would not exactly be a net win for the US, as well as potentially getting the agents of such an illegal approach arrested and tried.

Neither of these fantasies is going to come true. The US cannot simply remove Assange with no consequences, nor is he automatically cleared from another government’s decision to extradite him to the US, if the Attorney General charges him. This is going to take time and effort to work itself out.

Society is made up of competing goods that can’t be resolved in any perfect way — freedom vs. liberty, state secrets vs. citizen oversight — but the solutions to those tensions always take place in a particular context. Sometimes a bargain is so robust it lasts for centuries, as with trial by jury, but sometimes it is so much a product of its time that it does not survive the passing of its era.

I think that this latter fate has befallen our old balance between secrets and leaks. This does not mean that the Pentagon Papers precedent shouldn’t free Wikileaks from prosecution, but it does mean the old rules will not produce the old outcomes. In another difference with the Pentagon Papers case, Bradley Manning (or the person who copied the cables from SIPRnet) has gotten very little attention compared to Wikileaks, even though, in both law and practice, he is clearly the person most culpable. Like the music industry, the government is witnessing the million-fold expansion of edge points capable of acting on their own, without needing to ask anyone for help or permission, and, like the music industry, they are looking at various strategies for adding control at intermediary points that were left alone under the old model.

Julian claims that the history of these matters will be divided into “pre-” and “post-Cablegate” periods. This claim is grandiose and premature. However, it is not, on present evidence, visibly wrong.

It’s possible that the plain meaning of the Pentagon Papers case will clear Assange and Wikileaks, full stop, and the era of self-restraint of the press in response to extra-legal constraints is over, at least in the US context. It’s possible that the Pentagon Papers case will be re-adjudicated, and the press freedoms of the traditional press in the US will be dramatically constrained, relative to today. It’s possible that new laws will be written by Congress; it’s possible that those laws will be vetoed, or overturned, or amended. Whatever happens, though, this is new ground, and needs to be hashed out as an exemplar of the clash of basic principles that it is.

Like a lot of people, I am conflicted about Wikileaks.

Citizens of a functioning democracy must be able to know what the state is saying and doing in our name, to engage in what Pierre Rosanvallon calls “counter-democracy”*, the democracy of citizens distrusting rather than legitimizing the actions of the state. Wikileaks plainly improves those abilities.

On the other hand, human systems can’t stand pure transparency. For negotiation to work, people’s stated positions have to change, but change is seen, almost universally, as weakness. People trying to come to consensus must be able to privately voice opinions they would publicly abjure, and may later abandon. Wikileaks plainly damages those abilities. (If Aaron Bady’s analysis is correct, it is the damage and not the oversight that Wikileaks is designed to create.*)

And so we have a tension between two requirements for democratic statecraft, one that can’t be resolved, but can be brought to an acceptable equilibrium. Indeed, like the virtues of equality vs. liberty, or popular will vs. fundamental rights, it has to be brought into such an equilibrium for democratic statecraft not to be wrecked either by too much secrecy or too much transparency.

As Tom Slee puts it, “Your answer to ‘what data should the government make public?’ depends not so much on what you think about data, but what you think about the government.”* My personal view is that there is too much secrecy in the current system, and that a corrective towards transparency is a good idea. I don’t, however, believe in total transparency, and even more importantly, I don’t think that independent actors who are subject to no checks or balances is a good idea in the long haul.

If the long haul were all there was, Wikileaks would be an obviously bad thing. The practical history of politics, however, suggests that the periodic appearance of such unconstrained actors in the short haul is essential to increased democratization, not just of politics but of thought.

We celebrate the printers of 16th century Amsterdam for making it impossible for the Catholic Church to constrain the output of the printing press to Church-approved books*, a challenge that helped usher in, among other things, the decentralization of scientific inquiry and the spread of politically seditious writings advocating democracy.

This intellectual and political victory didn’t, however, mean that the printing press was then free of all constraints. Over time, a set of legal limitations around printing rose up, including restrictions on libel, the publication of trade secrets, and sedition. I don’t agree with all of these laws, but they were at least produced by some legal process.

Unlike the United States’ current pursuit of Wikileaks.*

I am conflicted about the right balance between the visibility required for counter-democracy and the need for private speech among international actors. Here’s what I’m not conflicted about: When authorities can’t get what they want by working within the law, the right answer is not to work outside the law. The right answer is that they can’t get what they want.

The Unites States is — or should be — subject to the rule of law, which makes the extra-judicial pursuit of Wikileaks especially nauseating. (Calls for Julian’s assassination are even more nauseating.) It may be that what Julian has done is a crime. (I know him casually, but not well enough to vouch for his motivations, nor am I a lawyer.) In that case, the right answer is to bring the case to a trial.

In the US, however, the government has a “heavy burden”, in the words of the Supreme Court, for engaging in prior restraint of even secret documents, an established principle since New York Times Co. vs. The United States*, when the Times published the Pentagon Papers. If we want a different answer for Wikileaks, we need a different legal framework first.

Though I don’t like Senator Joseph Lieberman’s proposed SHIELD law (Securing Human Intelligence and Enforcing Lawful Dissemination*), I do like the fact that it is a law, and not an extra-legal avenue (of which Senator Lieberman is also guilty.*) I also like the fact that the SHIELD Law makes it clear what’s at stake: the law proposes new restraints on publishers, and would apply to the New York Times and The Guardian as it well as to Wikileaks. (As Matthew Ingram points out, “Like it or not, Wikileaks is a media entity.”*) SHIELD amounts to an attempt to reverse parts of New York Times Co. vs. The United States.

I don’t think such a law should pass. I think the current laws, which criminalize the leaking of secrets but not the publishing of leaks, strike the right balance. However, as a citizen of a democracy, I’m willing to be voted down, and I’m willing to see other democratically proposed restrictions on Wikileaks put in place. It may even be that whatever checks and balances do get put in place by the democratic process make anything like Wikileaks impossible to sustain in the future.

The key, though, is that democracies have a process for creating such restrictions, and as a citizen it sickens me to see the US trying to take shortcuts. The leaders of Myanmar and Belarus, or Thailand and Russia, can now rightly say to us “You went after Wikileaks’ domain name, their hosting provider, and even denied your citizens the ability to register protest through donations, all without a warrant and all targeting overseas entities, simply because you decided you don’t like the site. If that’s the way governments get to behave, we can live with that.”

Over the long haul, we will need new checks and balances for newly increased transparency — Wikileaks shouldn’t be able to operate as a law unto itself anymore than the US should be able to. In the short haul, though, Wikileaks is our Amsterdam. Whatever restrictions we eventually end up enacting, we need to keep Wikileaks alive today, while we work through the process democracies always go through to react to change. If it’s OK for a democracy to just decide to run someone off the internet for doing something they wouldn’t prosecute a newspaper for doing, the idea of an internet that further democratizes the public sphere will have taken a mortal blow.

It is, perhaps, the end of the beginning.

In early July, Rupert Murdoch’s News Corporation placed its two London-based “quality” dailies, the Times and Sunday Times, behind a paywall, charging £1 for 24 hours access, or £2 a week (after an introductory £1 for the first month.*) At the same time, News Corp also forbad the UK’s Audit Bureau of Circulations from reporting site traffic*, so that no meaningful measure of the paywall’s effect was available.

That situation has now been partially reversed, with News reporting some of its own numbers: they claim 105,000 total transactions for digital content between July and October.* (Several people have wrongly reported this as 105,000 users. The number of users is smaller, as there can be more than one transaction per user.) News Corp notes that about half of those transactions were one-offs, meaning only about 50,000 transactions in those four months were by people with any commitment to the site longer than a single day.

Because that 50K number includes not just web site transactions, but Kindle and iPad sales as well, web subscribers are, at best, in the low tens of thousands. However, we don’t know how small the digital subscriber number is, for two reasons. First, the better the Kindle and iPad numbers are, the worse the web numbers are. Second, News did not report, for example, whether a loyal reader from July to October would count as a single transaction or several consecutive transactions. (If iPad sales are good, and loyal users create multiple transactions, then monthly web subscribers could be under 10,000.)

The other figure News reported is that something like 100,000 print subscribers have requested web access. Combining digital-only and print subscribers, and comparing them with comScore’s pre-paywall estimate of roughly six million unique readers worldwide*, the reduction in total web audience seems to be on the order of 97%. (Note that this reduction can’t be measured by before and after traffic, as the home pages are outside the paywall, so people who refuse to pay still show up as visitors.)

Because the print subscribers outnumber digital-only users, most of the remaining 3% pay nothing for the site. Subscription to the paper is now a better draw for website use than any case News has been able to make for paid access.

Given the paucity of the data, the key question of churn remains unanswerable. After the introductory £1 a month offer, the annualized rate rises from £12 to £104. This will cause additional users to bail out, but we have no way of guessing how many.

As with every aspect of The Times’ paywall, interpretation of these numbers varies widely. There are people arguing that these numbers are good news; Robert Andrews at PaidContent sees hope in the Times now having recurring user revenues.* There are people arguing that they are bad news; Mike Masnick at TechDirt believes those revenues are unlikely to offset new customer acquition costs and the loss of advertising.* What is remarkable though, what seems to need more explaining than News’s strategy itself, is why anyone regards this particular paywall as news at all.

The “paywall problem” isn’t particularly complex, either in economic or technological terms. General-interest papers struggle to make paywalls work because it’s hard to raise prices in a commodity market. That’s the problem. Everything else is a detail.

The classic description of a commodity market uses milk. If you own the only cow for 50 miles, you can charge usurious rates, because no one can undercut you. If you own only one of a hundred such cows, though, then everyone can undercut you, so you can’t charge such rates. In a competitive environment like that, milk becomes a commodity, something whose price is set by the market as a whole.

Owning a newspaper used to be like owning the only cow, especially for regional papers. Even in urban markets, there was enough segmentation–the business paper, the tabloid, the alternative weekly–and high enough costs to keep competition at bay. No longer.

The internet commodifies the business of newspapers. Any given newspaper competes with a few other newspapers, but any newspaper website compete with all other websites. As Nicholas Carr pointed out during the 2009 pirate kidnapping, Google News found 11,264 different sources for the story, all equally accessible.* The web puts newspapers in competition with radio and TV stations, magazines, and new entrants, both professional and amateur. It is the war of each against all.

None of this is new. The potential disruptive effects of the internet on newspapers have been observable since ClariNet in 1989.* Nor has the business case for paywalls changed. The advantage of paywalls is that they raise revenue from users. The disadvantages are that they reduce readership, increase customer acquistion and retention costs, and eliminate ad revenue from user-forwarded content. In most cases, the disadvantages have outweighed the advantages.

So what’s different about News paywall? Nothing. It’s no different from other pay-for-access plans, whether the NY Times’ TimesSelect* or the Harligen Texas Valley Morning Star.* News Corp has produced no innovation in content, delivery, or payment, and the idea of 90%+ loss of audience was already a rule of thumb over a decade ago. Yet something clearly feels different.

Over the last fifteen years, many newspaper people have assumed continuity with the analog business model, which is to say they assumed that readers could eventually be persuaded or forced pay for digital editions. This in turn suggested that the failure of any given paywall was no evidence of anything other than the need to try again.

What is new about the Times’ paywall–what may in fact make it a watershed–isn’t strategy or implementation. What’s new is that it has launched as people in the news business are re-thinking assumed continuity. It’s new because the people paying attention to it are now willing to regard the results as evidence of something. To the newspaper world, TimesSelect looked like an experiment. The Times and Sunday Times look like a referendum on the future.

One way to escape a commodity market is to offer something that isn’t a commodity. This has been the preferred advice of people committed to the re-invention of newspapers. It is a truism bordering on drinking game material that anyone advising newspapers will at some point say “All you need to do is offer a product so relevant and valuable the consumer is willing to pay for it!”

This advice is well-meaning. It’s just not much help. The suggestion that newspapers should, in the future, create a digital product users are willing to pay for is merely a restatement of the problem, by way of admission that the current product does not pass that test.

Most of the historical hope for paywalls assumed that through some combination of reader desire and supplier persuasiveness, the current form of the newspaper could survive the digital transition without significant alteration.

Payalls, as actually implemented, have not accomplished this. They don’t expand revenue from the existing audience, they contract the audience to that subset willing to pay. Paywalls do indeed help newspapers escape commodification, but only by ejecting the readers who think of the product as a commodity. This is, invariably, most of them.

You can see this contraction at the Times and Sunday Times in the reversal of digital to print readers. Before the paywall, the two sites had roughly six times more readers than there were print sales of the paper edition. (6M web vs. 1M print for the Sunday Times* .) Post-paywall, the web audience is less than a sixth of print sales (down to <150K vs. 1M). The paying web audience is less a twentieth of print sales (<50K vs. 1M), and possibly much less.

One way to think of this transition is that online, the Times has stopped being a newspaper, in the sense of a generally available and omnibus account of the news of the day, broadly read in the community. Instead, it is becoming a newsletter, an outlet supported by, and speaking to, a specific and relatively coherent and compact audience. (In this case, the Times is becoming the online newsletter of the Tories, the UK’s conservative political party, read much less widely than its paper counterpart.)

Murdoch and News Corp, committed as they have been to extracting revenues from the paywall, still cannot execute in a way that does not change the nature of the organizations behind the wall. Rather than simply shifting relative subsidy from advertisers to users for an existing product, they are instead re-engineering the Times around the newsletter model, because the paywall creates newsletter economics.

As of July, non-subscribers can no longer read Times stories forwarded by colleagues or friends, nor can they read stories linked to from Facebook or Twitter. As a result, links to Times stories now rarely circulate in those media. If you are going to produce news that can’t be shared outside a particular community, you will want to recruit and retain a community that doesn’t care whether any given piece of news spreads, which means tightly interconnected readerships become the ideal ones. However, tight interconnectedness correlates inversely with audience size, making for a stark choice, rather than offering a way of preserving the status quo.

This re-engineering suggests that paywalls don’t and can’t rescue current organizational forms. They offer instead yet another transformed alternative to it. Even if paywall economics can eventually be made to work with a dramatically reduced audience, this particular referendum on the future (read: the present) of newspapers is likely to mean the end of the belief that there is any non-disruptive way to remain a going concern.

I’ve bundled some replies to various questions in the comments from November 9th here and from November 10th here.

Also, nota bene: One of the problems with the various “Hey you guys, I just had a great idea for saving newpapers!” micropayment comments showing up in my moderation queue is that the proposers often exhibit no understanding that micropayments have a 20-year history of failure.

I will not post comments suggesting that micropayments will save the news industry unless those comment refer to at least some of the theoretical or practical literature on previous attempts to make them work for the news business. Start here: Why Small Payments Won’t Save Publishers