Clay Shirky's Essays

VoIP - Plan A vs Plan B

2003 was a remarkable year in the US for voice over the internet (VoIP). If you needed a label for the events of the year, "Collapse of Denial" would be a good one -- after a long period of relative inaction, the FCC and the state regulators are suddenly pushing hard for a regulatory framework. The question is no longer whether voice is going to become an internet application, but when. "When" could still be a very long time, however. The incumbent local phone companies -- Verizon, SBC, BellSouth and Qwest -- have various degrees of interest in VoIP, but are loathe to embrace it quickly or completely, because doing so means admitting to everyone -- shareholders, regulators, customers -- that both monopoly control and artificially high voice revenues are going away. (The fact that this is true does not much lessen the pain of saying so.) As a result, they will likely try to convince regulatory agencies, both the FCC and the states', to burden competitive VoIP firms like Vonage with additional costs and rules, while delaying their own offerings. Complicating this de facto Plan A, however, is the fact that VoIP isn't a service, it's just a set of protocols, meaning that competitors don't have to buy into Plan A to deploy it. If Plan A is "Replace the phone system slowly and from within," Plan B is far more radical: "Replace the phone system. Period." - More at http://www.shirky.com/writings/voip_a_b.txt

The Possibility of Spectrum as a Public Good

The FCC is considering opening up additional spectrum to unlicensed uses -- the same kind of regulatory change that gave rise to Wifi. Much of the spectrum being considered for unlicensed use is currently allocated for broadcasters, however, so FCC's proposal creates tension between incumbents and groups that want to take advantage of the possibilities inherent in unlicensed spectrum. Most issues the FCC deals with, even contentious ones like limits on the ownership of radio and television stations, are changes within regulatory schemes. The recent proposal to move the maximum media market reach from 35% to 45% took the idea of an ownership cap itself at face value, and involved a simple change of amount. Unlicensed spectrum is different. In addition to all the regulatory complexities, an enormous philosophical change is being proposed. Transmuting spectrum from licensed to unlicensed changes what spectrum is. This change is possible because of advances in the engineering of wireless systems. This matters, a lot, because with the spread of unlicensed wireless, the FCC could live up to its mandate of managing spectrum on behalf of the public, by allowing for and even encouraging engineering practices that treat spectrum itself as a public good. - More at http://shirky.com/writings/spectrum_public_good.html

The Semantic Web, Syllogisms, and Worldview

The W3C's Semantic Web project has been described in many ways over the last few years: "an extension of the current web in which information is given well-defined meaning" [1], "a place where machines can analyze all the data on the Web" [2], even "a Web in which machine reasoning will be ubiquitous and devastatingly powerful." [3] The problem with descriptions this general, however, is that they don't answer the obvious question: What is the Semantic Web good for? The simple answer is this: The Semantic Web is a machine for creating syllogisms. A syllogism is a form of logic, first described by Aristotle, where "...certain things being stated, something other than what is stated follows of necessity from their being so." The Semantic Web specifies ways of exposing assertions on the Web, so that third parties can combine them to discover things that are true but not specified directly. This is the promise of the Semantic Web -- it will improve all the areas of your life where you currently use syllogisms. Which is to say, almost nowhere. - More at http://www.shirky.com/writings/semantic_syllogism.html

Is Social Software Bad for the Dean Camapign?

I'm getting the same cognitive dissonance listening to political handicappers explain Dean's dismal showing in Iowa that I used to get listening to financial analysts try to explain dot com mania with things like P/E ratios and EBITDA. A stock's value is not set by those things; it is set by buyer and seller agreeing on price. In ordinary markets, buyers and sellers use financial details to get to that price, but sometimes, as with dot com stocks, the way prices get agreed on has nothing to do with finance. In the same way, talking about Dean's third-place showing in terms of 'momentum' and 'character', the P/E and EBITDA of campaigns, may miss the point. Dean did poorly because not enough people voted for him, and the usual explanations -- potential voters changed their minds because of his character or whatever -- seem inadequate to explain the Iowa results. What I wonder is whether Dean has accidentally created a movement (where what counts is believing) instead of a campaign (where what counts is voting.) And (if that's true) I wonder if his use of social software helped create that problem. - More at http://www.corante.com/many/archives/2004/01/26/is_social_software_bad_for_the_dean_campaign.php

The RIAA Succeeds Where the Cypherpunks Failed

For years, the US Government has been terrified of losing surveillance powers over digital communications generally, and one of their biggest fears has been broad public adoption of encryption. If the average user were to routinely encrypt their email, files, and instant messages, whole swaths of public communication currently available to law enforcement with a simple subpoena (at most) would become either unreadable, or readable only at huge expense. [...] The Government's failure to get the Clipper implemented came at a heady time for advocates of digital privacy -- the NSA was losing control of cryptographic products, Phil Zimmerman had launched his Pretty Good Privacy (PGP) email program, and the Cypherpunks, a merry band of crypto-loving civil libertarians, were on the cover of [http://www.wired.com/wired/archive/1.02/crypto.rebels.html] the second issue of Wired. The floodgates were opening, leading to... ...pretty much nothing. Even after the death of Clipper and the launch of PGP, the Government discovered that for the most part, users didn't _want_ to encrypt their communications. The single biggest barrier to the spread of encryption has turned out to be not control but apathy. Though business users encrypt sensitive data to hide it from one another, the use of encryption to hide private communications from the Government has been limited mainly to techno-libertarians and a small criminal class. - More at http://www.shirky.com/writings/riaa_encryption.html

Permanet, Nearlynet, and Wireless Data

For most of the past year, on many US airlines, those phones inserted into the middle seat have borne a label reading "Service Disconnected." Those labels tell a simple story -- people don't like to make $40 phone calls. They tell a more complicated one as well, about the economics of connectivity and about two competing visions for access to our various networks. One of these visions is the one everyone wants -- ubiquitous and convenient -- and the other vision is the one we get -- spotty and cobbled together.

Call the first network "perma-net," a world where connectivity is like air, where anyone can send or receive data anytime anywhere. Call the second network "nearly-net", an archipelago of connectivity in an ocean of disconnection. Everyone wants permanet -- the providers want to provide it, the customers want to use it, and every few years, someone announces that they are going to build some version of it. The lesson of in-flight phones is that nearlynet is better aligned with the technological, economic, and social forces that help networks actually get built. The most illustrative failure of permanet is the airphone. The most spectacular was Iridium. The most expensive will be 3G. - More at http://www.shirky.com/writings/permanet.html

Situated Software

I teach at NYU's Interactive Telecommunications Program (ITP), where the student population is about evenly divided between technologists who care about aesthetics and artists who aren't afraid of machines, which makes it a pretty good place to see the future. Part of the future I believe I'm seeing is a change in the software ecosystem which, for the moment, I'm calling situated software. This is software designed in and for a particular social situation or context. This way of making software is in contrast with what I'll call the Web School (the paradigm I learned to program in), where scalability, generality, and completeness were the key virtues. I see my students cheerfully ignoring Web School practices and yet making interesting work, a fact that has given me persistent cognitive dissonance for a year, so I want to describe the pattern here, even in its nascent stages, to see if other people are seeing the same thing elsewhere. - More at http://www.shirky.com/writings/situated_software.html

Ontology is Overrated: Categories, Links, and Tags

Today I want to talk about categorization, and I want to convince you that a lot of what we think we know about categorization is wrong. In particular, I want to convince you that many of the ways we're attempting to apply categorization to the electronic world are actually a bad fit, because we've adopted habits of mind that are left over from earlier strategies. I also want to convince you that what we're seeing when we see the Web is actually a radical break with previous categorization strategies, rather than an extension of them. The second part of the talk is more speculative, because it is often the case that old systems get broken before people know what's going to take their place. (Anyone watching the music industry can see this at work today.) That's what I think is happening with categorization. What I think is coming instead are much more organic ways of organizing information than our current categorization schemes allow, based on two units -- the link, which can point to anything, and the tag, which is a way of attaching labels to links. The strategy of tagging -- free-form labeling, without regard to categorical constraints -- seems like a recipe for disaster, but as the Web has shown us, you can extract a surprising amount of value from big messy data sets. - More at http://shirky.com/writings/ontology_overrated.html

Nomic World: By the players, for the players

[This is an edited version of the talk I gave last fall at the State of Play conference.] I'm sort of odd-man-out in a Games and Law conference, in that my primary area of inquiry isn't games but social software. Not only am I not a lawyer, I don't even spend most of my time thinking about game problems. I spend my time thinking about software that supports group interaction across a fairly wide range of social patterns. So, instead of working from case law out, which has been a theme here (and here's where I insert the "I am not a lawyer" disclaimer) I'm going to propose a thought experiment looking from the outside in. And I want to pick up on something that Julian [Dibbell] said earlier about game worlds: 'users are the state.' The thought experiment I want to propose is to agree with that sentiment, and to ask "How far can we go in that direction?" Instead of looking for the places where game users are currently suing or fighting one another, forcing the owners of various virtual worlds to deal with these things one crisis at a time, I want to ask the question "What would happen if we wanted to build a world where we maximized the amount of user control? What would that look like?" I'm going to make that argument in three pieces. First, I'm going to do a little background on group structure and the tension between the individual and the group. Then I want to contrast briefly governance in real and virtual worlds. Finally I want to propose a thought experiment on placing control of online spaces in the hands of the users. - More at http://www.shirky.com/writings/nomic.html

My new book, "Here Comes Everybody", is out.

My new book is out. - More at http://HereComesEverybody.org

Group As User: Flaming and the Design of Social Software

When we hear the word "software," most of us think of things like Word, Powerpoint, or Photoshop, tools for individual users. These tools treat the computer as a box, a self-contained environment in which the user does things. Much of the current literature and practice of software design -- feature requirements, UI design, usability testing -- targets the individual user, functioning in isolation. And yet, when we poll users about what they actually do with their computers, some form of social interaction always tops the list -- conversation, collaboration, playing games, and so on. The practice of software design is shot through with computer-as-box assumptions, while our actual behavior is closer to computer-as-door, treating the device as an entrance to a social space. We have grown quite adept at designing interfaces and interactions between computers and machines, but our social tools -- the software the users actually use most often -- remain badly misfit to their task. Social interactions are far more complex and unpredictable than human/computer interaction, and that unpredictability defeats classic user-centric design. As a result, tools used daily by tens of millions are either ignored as design challenges, or treated as if the only possible site of improvement is the user-to-tool interface. The design gap between computer-as-box and computer-as-door persists because of a diminished conception of the user. The user of a piece of social software is not just a collection of individuals, but a group. Individual users take on roles that only make sense in groups: leader, follower, peacemaker, process nazi, and so on. There are also behaviors that can only occur in groups, from consensus building to social climbing. And yet, despite these obvious differences between personal and social behaviors, we have very little design practice that treats the group as an entity to be designed for. There is enormous value to be gotten in closing that gap, and it doesn't require complicated new tools. It just requires new ways of looking at old problems. Indeed, much of the most important work in social software has been technically simple but socially complex. - More at http://shirky.com/writings/group_user.html

Fame vs Fortune: Micropayments and Free Content

Micropayments, small digital payments of between a quarter and a fraction of a penny, made (yet another) appearance this summer with Scott McCloud's online comic, The Right Number, accompanied by predictions of a rosy future for micropayments. To read The Right Number, you have to sign up for the BitPass micropayment system; once you have an account, the comic itself costs 25 cents.

BitPass will fail, as FirstVirtual, Cybercoin, Millicent, Digicash, Internet Dollar, Pay2See, and many others have in the decade since Digital Silk Road, the paper that helped launch interest in micropayments. These systems didn't fail because of poor implementation; they failed because the trend towards freely offered content is an epochal change, to which micropayments are a pointless response.

The failure of BitPass is not terribly interesting in itself. What is interesting is the way the failure of micropayments, both past and future, illustrates the depth and importance of putting publishing tools in the hands of individuals. In the face of a force this large, user-pays schemes can't simply be restored through minor tinkering with payment systems, because they don't address the cause of that change -- a huge increase the power and reach of the individual. - More at http://www.shirky.com/writings/fame_vs_fortune.html

A Group Is Its Own Worst Enemy

(This essay is a lightly edited version of a talk, "A Group Is Its Own Worst Enemy", about persistent patterns in the design and operation of large-scale and long-lived online groups. I gave this talk at the O'Reilly Emerging Tech conference in April of 2003.) This talk is in three parts. The best explanation I have found for the kinds of things that happen when groups of humans interact is psychological research that predates the Internet, so the first part is going to be about W.R. Bion's research, which I will talk about in a moment, research that I believe explains how and why a group is its own worst enemy. The second part is: Why now? What's going on now that makes this worth thinking about? I think we're seeing a revolution in social software in the current environment that's really interesting. And third, I want to identify some things, about half a dozen things, in fact, that I think are core to any software that supports larger, long-lived groups. - More at http://www.shirky.com/writings/group_enemy.html

File-sharing Goes Social

The RIAA has taken us on a tour of networking strategies in the last few years, by constantly changing the environment file-sharing systems operate in. In hostile environments, organisms often adapt to become less energetic but harder to kill, and so it is now. With the RIAA's waves of legal attacks driving experimentation with decentralized file-sharing tools, file-sharing networks have progressively traded efficiency for resistance to legal attack.

The RIAA has slowly altered the environment so that relatively efficient systems like Napster were killed, opening up a niche for more decentralized systems like Gnutella and Kazaa. With their current campaign against Kazaa in full swing, we are about to see another shift in network design, one that will have file sharers adopting tools originally designed for secure collaboration in a corporate setting. - More at http://www.shirky.com/writings/file-sharing_social.html

Social Software and the Politics of Groups

Social software, software that supports group communications, includes everything from the simple CC: line in email to vast 3D game worlds like EverQuest, and it can be as undirected as a chat room, or as task-oriented as a wiki (a collaborative workspace). Because there are so many patterns of group interaction, social software is a much larger category than things like groupware or online communities -- though it includes those things, not all group communication is business-focused or communal. One of the few commonalities in this big category is that social software is unique to the internet in a way that software for broadcast or personal communications are not. - More at http://www.shirky.com/writings/group_politics.html

Grid Computing: The Next Push

Grid Computing is, according to the Grid Information Centre, a way to "...enable the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources." It is, in other words, an attempt to make Sun's famous pronouncement "The Network Is The Computer" an even more workable proposition. (It is also an instantiation of several of the patterns of decentralization that used to travel together under the name peer-to-peer.)

Despite the potential generality of the Grid, most of the public pronouncements are focusing on the use of Grids for supercomputing. IBM defines it more narrowly: Grid Computing is "... applying resources from many computers in a network-at the same time-to a single problem", and the MIT Technology Review equated Grid technology with supercomputing on tap when it named Grids one of "Ten Technologies That Will Change the World."

This view is wrong. Supercomputing on tap won't live up to to this change-the-world billing, because computation isn't a terribly important part of what people do with computers. This is a lesson we learned with PCs, and it looks like we will be relearning it with Grids. - More at http://www.shirky.com/writings/grids.html

The FCC, Weblogs, and Inequality

Yesterday, the FCC adjusted the restrictions on media ownership, allowing newspapers to own TV stations, and raising the ownership limitations on broadcast TV networks by 10%, to 45% from 35%. It's not clear whether the effects of the ruling will be catastrophic or relatively unimportant, and there are smart people on both sides of that question. It is also unclear what effect the internet had on the FCC's ruling, or what role it will play now.

What is clear, however, is a lesson from the weblog world: inequality is a natural component of media. For people arguing about an ideal media landscape, the tradeoffs are clear: Diverse. Free. Equal. Pick two. - More at http://shirky.com/writings/fcc_inequality.html