Clay Shirky's Writings About the Internet

Economics & Culture, Media & Community, Open Source

View Source... Lessons from the Web's massively parallel development.

First published April 1998.

How did Web grow so quickly?

In the 36 months from January of 1993 to December of 1995, HTML went from being an unknown protocol to being to being the pre-eminent tool for designing electronic interfaces, decisively displacing almost all challengers and upstaging online services, CD-ROMs, and a dozen expensive and abortive experiments with interactive TV, and it did this while having no coordinated center, no central R&D effort, and no discernible financial incentive for the majority of its initial participants.

Ragtag though it was, the early Web, with its endless compendia of 'Cool Bands', badly scanned photos of pets, and ubiquitous "Favorite Links" lists pointing to more of same, was a world-building excercise which provided an unequalled hothouse lab for structuring information and designing interfaces. By the time the corporate players and media companies considered getting involved, many of the basic interface issues - the idea of Home pages, buttons and toolbars, the standards for placement of things like logos, navigational elements, help menus and the like had already been worked out within a vast body of general practice generated by the pioneers.

More than any other factor, this ability to allow ordinary users to build their own Web sites, without requiring that they be software developers or even particularly savvy software users, caused its rapid rise. Web design, and in particular Web site design with its emphasis on architecture, interactivity, and structuring data in an implicit order or set of orders, is not graphic design but rather constitutes a special kind of low-level engineering. Prior to the invention of HTML, all software packages which implemented any sort of hypertext required the user to learn a programming language of some sort (Director, Hypercard). HTML, in contrast, allows users to create links without requiring an ablity to write code.

INTERFACE ENGINEERING

Call HTML's practitioners interface designers, information architects or, in keeping with the idea of world-building, even citizen-engineers, these are people who can think creatively about information as media, without necessarily being comfortable looking down the business end of a compiler. This becomes Principle 0 for this essay:

#0. Web site design is a task related to, but not the same as, software engineering.

Any project which offers a limitless variety of ways in which information can be presented and stuctured can happen both faster and better if there is a way for designers and engineers to collaborate without requiring either group to fundamentally alter their way looking at the world. While it can't hurt to have a designer see source code or a developer open a copy of Photoshop once in a while, environments which let each group concentrate on their strengths grow more rapidly than environments which burden designers with engineering concerns or force their engineers to design an interface whenever they want to alter an engine. The Web exploded in the space of a few years in part because it effected this separation of functions so decisively.

As opposed to software which seeks a tight integration between creation, file format, and display (e.g. Excel, Lotus Notes), HTML specifies nothing about the tools needed for its creation, validation, storage, delivery, interpretation, or display. By minimizing the part of the interface that is the browser to a handful of commands for navigating the Web, while maximizing the part of the interface that is in the browser (the HTML files themselves), the earliest experiments with the Web took most of the interface out of the browser code and put it in the hands of any user who could learn HTML's almost babyish syntax.

With Web design separated from the browser that was displaying it, countless alternative ways of structuring a site could be experimented with - sites for online newspapers, banks, stores, magazines, interfaces for commerce, document delivery, display, propaganda, art - all without requiring that the designers arrange their efforts with the browser engineers or even with one another.

This separation of interface and engineering puts site design in a fundamentally new relationship to the software that displays it:

#1. An interface can be integrated with the information it is displaying, instead of with the display software itself.

This transfer of the interface from 'something that resides in the software and is applied to the data' to 'something that resides in the data and is applied to the software' is the single most important innovation of the early Web.

REDUCING REQUIRED COORDINATION

Site designers with training in visual or conceptual aspects of organizing and presenting information can design for a Web browser without working with, or even needing to talk to, the people who designed the browser itself. Newsweek has to coordinate with America Online in order to make its content available through AOL, but it does not have to coordinate with Netscape (or the end-user's ISP) to make its content available through Netscape.

This move away from centralized coordination and towards lateral development is good for the Web, and that some basic principles about the ways this is acheived can be derived from looking at the HTML/Browser split as one instance of a general class of "good" tools. The basic rule of good tools is:

#2. Good tools are transparent.

Web design is a conversation of sorts between designers, with the Web sites themselves standing in for proposal and response. Every page launched carries an attached message from its designer which reads "I think this is a pretty good way to design a Web page", and every designer reacting to that will be able to respond with their own work. This is true of all design efforts - cars, magazines, coffee pots, but on the Web, this conversation is a riot, swift and wide ranging.

THE CENTRALITY OF OPEN HTML SOURCE TO THE WEB'S SUCCESS

The single factor most responsible for this riot of experimentation is transparency - the ability of any user to render into source code the choices made by any other designer. Once someone has worked out some design challenge, anyone else should be able to adopt, modify it, and make that modified version available, and so on.

Consider how effortless it would have been for Tim Berners-Lee or Marc Andreeson to treat the browser's "View Source..." as a kind of debugging option which could have been disabled in any public release of their respective browsers, and imagine how much such a 'hidden source' choice would have hampered this feedback loop between designers. Instead, with this unprecedented transparency of the HTML itself, we got an enormous increase in the speed of design development. When faced with a Web page whose layout or technique seems particularly worth emulating or even copying outright, the question "How did they do that?" can be answered in seconds.

"X DOES Y", NOT "X IS FOR Y"

This ability to coordinate laterally, for designers to look at one anothers work and to experiment with it without asking permission from a central committee, is critical to creating this speed of innovation. Once the tools for creating Web pages are in a designers hand, there is no further certification, standardization or permission required. Put another way:

#3. Good tools tell you what they do, not what they're for.

Good, general-purpose tools specify a series of causes and effects, nothing more. This is another part of what allowed the Web to grow so quickly. When a piece of software specifies a series of causes and effects without specifying semantic values (gravity makes things fall to the ground, but gravity is not for keeping apples stuck to the earth's surface, or for anything else for that matter), it maximises the pace of innovation, because it minimizes the degree to which an effect has to be planned in advance for it to be useful.

The best example of this was the introduction of tables, first supported in Netscape 1.1. Tables were originally imagined to be just that - a tool for presenting tabular data. Their subsequent adoption by the user community as the basic method for page layout did not have to be explicit in the design of either HTML or the browser, because once its use was discovered and embraced, it no longer mattered what tables were originally for, since they specified causes and effects that made them perfectly suitable in their new surroundings.

A corrolary to rule #3 is:

#3b. Good tools allow users to do stupid things.

A good tool, a tool which maximizes the possibilities for unexpected innovation from unknown quarters, has to allow the creation of everything from brilliant innovation through workmanlike normalcy all the way through hideous dreck. Tools which try to prevent users from making mistakes enter into a tar pit, because this requires that in addition to cause and effect, a tool has to be burdened with a second, heuristic sense of 'right' and 'wrong'. In the short run, average quality can be raised if a tool intervenes to prevent legal but inefficent uses, but in the long haul, that strategy ultimately hampers development by not letting users learn from their mistakes.

THE INDEPENDENT RATE OF DEVLOPMENT

The browser/HTML combination as described not only increases the number of people who can work on different functions by reducing the amount of cross-training and coordination needed between designers and engineers (or between any two designers), it also changes the speed at which things happen by letting designers develop different parts of the system at independant rates.

By separating Web site design so completely from browser engineering, the rate of development of Web sites became many times faster than the rate of browser-engineering or protocol design.

Reflecting on these extremes is instructive, considered in light of the traditional software release schedule, which has alterations to engineering and interface change appearing together, in relatively infrequent versions. In the heat of structuring a site, a Web design team may have several people altering and re-examining an interface every few minutes. At the opposite extreme, in the three years from 1993 to 1995, the http protocol was not changed once, and in fact, even in 1998, the majority of Web transactions still use http 1.0. In between these extremes come rates of change from more to less rapid: changes in site structure (the relation between file storage and pointers), new versions of Web browsers, and new specifications for HTML itself. Designers could create, display, alter, beg, borrow, and steal interfaces, as fast as they liked, without any further input from any other source.

CONCLUSION

The Web grew as quickly as it did because the independent rate of site design, freed from the dictates of browser engineering, was much faster than even its inventors had predicted. Tools that allowed designers to do anything that rendered properly, that allowed for lateral conversations through the transparency of the HTML source, and removed the need for either compiling the results or seeking permission, certifcation or registration from anyone else led to the largest example of parallel development seen to date, and the results have been world-changing.

Furthermore, while there were certainly aspects of that revolution which will not be easily repeated, there are several current areas of inquiry - multi-player games (e.g. Half Life, Unreal), shared 3D worlds (VRML, Chrome), new tagset proposals (XML, SMIL), new interfaces (PilotOS, Linux), which will benefit from examination in light of the remarkable success of the Web. Any project with an interface likely to be of interst to end users (create your own avatar, create your own desktop) can happen both faster and better if these principles are applied.

First published April 1998.

Clay Shirky's Writings About the Internet

Economics & Culture, Media & Community, Open Source