shirky.com Clay Shirky's Writings About the Internet
Economics and Culture, Media and Community, Open Source

"The Interest Horizons and the Limits of Software Love"


Commercial software has its limits, and it runs into them regularly.
Any commercial developer has a "resource horizon" - some upper
boundary of money or programmer time which limits how much can be done
on any given project. Projects which lie over this horizon either
can't be accomplished (very few companies could write a new operating
system from scratch), or, once started, can't be completed because of
their size or scope (as with the Denver Airport baggage handling
system).

Open Source software has no such resource horizon - programmers
creating Open Source software do it because they're interested in it,
not because they're paid to do it. Furthermore, since there's no
requirement to keep the source code secret, a much larger of number of
programmers can work on any given project, and they don't have to work
for the same company, or even live in the same country. 

However, the lack of this resource horizon does not mean that Open
Source software has no limits, it just means that the limits aren't
financial.  Commercial companies make software for money, so money is
the limiting factor. Open Source developers make software for the love
of the thing, so love becomes the limiting factor as well. Unloved
software can't be built using Open Source methods.

Open Source software trades the resource horizon for an interest
horizon, the horizon over which a problem simply isn't interesting
enough to attract a group of developers who burn to spend their
Friday nights wrestling with, say, meter-reading software for a
natural gas pipeline. Consider, from the seminal description of the
Open Source process contained in 
"The Cathedral and the Bazaar" , how many of the principles assume
that programmers find problems, rather than vice-versa:

  Every good work of software starts by scratching a developer's
  personal itch. 
...
  If you have the right attitude, interesting problems will find you.
...
  Often, the most striking and innovative solutions come from
  realizing that your concept of the problem was wrong.
...
  To solve an interesting problem, start by finding a problem that
  is interesting to you.
...
  Given a large enough beta-tester and co-developer base, almost every
  problem will be characterized quickly and the fix obvious to
  someone.

This is geek nirvana, where interest and innovation have more weight
than profit or productivity. Because the articulation of these
principles is so new (though the way of working is at least as old as
the Internet itself), and because the acheivements of Linux, Apache et
al. are so unprecedented, Open Source currently has an aura of
unstoppability, a sense that "Maybe Open Source will take over
everything!"

It won't. Wherever Open Source is a good way of working, it is more
effective than closed source, for the same reason that open markets
are more effective than central planning. The very principles of
personal interest outlined above, however, indicate where Open Source
_isn't_ a good way of working in the first place.

Consider the converse of those principles: what if you have a problem
that _doesn't_ scratch some core developer's personal itch?

  "I stayed up all weekend rewriting this awesome time-sheet
  processing system!"

or where innovative solutions _aren't_ an option in an existing system?

  "We could make the package much simpler if we just bill them every
  year instead of every month."

or where you _can't_ have a large enough group of beta testers?

  "Know anybody who owns a plane? I need to beta-test my air traffic
  control program."

These kinds of problems lie beyond the interest horizon, and their
numbers are legion. 

Existing software which is old or broken but has to be updated or
fixed won't clear the "interesting problem" hurdle. Boring software
won't acquire enough beta testers. Projects which require treating the
user like an idiot (an oxymoron in the Open Source world, since
scratching a personal itch turns the lead developer into User #1)
won't attract enough developers to take off. The overlapping
categories of old, broken, boring and disdainful of the user's
intelligence currently describe a majority of the world's software, an
installed base that isn't going away any time soon.

Given this interest horizon, what Open Source can do is create a
market for software as service rather than as product. One way to get
people to do things they wouldn't do for love is to pay them. Who
wants to sit at a desk all day, shuffle paper and answer phones as a
hobby? Throw in $25 Gs and health insurance to sweeten the deal,
though, and the answer is "A lot of people". The same will be true of
the interest horizon.  There will never be an Open Source billing
system for a water utility, but there will be Open Source accounting
and report-generating packages which provide about 80% of the
functionality, meaning that the water works only needs to do pay for
the last 20% of the work, not all of it.

30 years ago, every IT department in this country was in the business
of building custom products, and the software industry grew up around that
assumption. Everyone solved their own problems in-house, leading to
vast duplication of effort as thousands of nearly identical record
keeping systems, billing systems and databases were created from
scratch, over and over again.

15 years ago the rise of high-quality shrink-wrapped software began to
change that, where people could buy products which had some core set
of functionality which they could extend. Problems were no longer
solved from scratch, but a huge pile of money still went to pay for
the identical core functionality within each business.

Now, Open Source suggests an almost pure service model, where the
basic functionality costs nothing, and all the money is in
customization.  Imagine an economy where the basic tools and serivces
are free: in this world, once a company it knew what it wanted to
build, it could put the basic software functionality in place for free
and then either hire or outsource the talent to do whatever
customizations were necessary.

This economy exists today, on the Web. Even a simple web site is a
complicated piece of software, yet millions of individuals and small
companies run Web sites, because the basic functionality of the Web
server doesn't need to be recreated from scratch every time. More
surprising still, the Web does this so well that we hardly give it a
moment's thought.

Yet consider how far Open Source has gone to make this possible. In
the 80's, CompuServe, Prodigy and AOL all had the same rough set of
goals and features, yet each of them had to build, from scratch, a
data network, a set of interactive applications, and a software
interface. It would have benefitted each of them enormously if they
could have inherited basic functions, and then concentrated on their
particular strengths, but there was no way in those days to do
anything of the sort.

Now consider Yahoo, Geocities and Hotmail. All three use the same Open
Source operating system (FreeBSD) and the same Open Source Web server
(Apache), which means that their programmers and designers have been
free to focus on making Yahoo be Yahoo, Geocities be Geocities, and
Hotmail Hotmail - customization from the jump.  From the same core of
free functionality - net-connected operating system, web server, html
interface, interactive scripts - we get three different sites, with
three different interfaces, functions, business models, and audiences,
yet each company spent much less money and energy, for much more gain,
than the online services of old.

This is the future of Open Source. As the edges of the interest
horizon become clear, the market for products will shrink and the
market for customization will grow: Expect the Computer Associates of
the world to begin training their employees in the use and
customization of Open Source products. With Open Source software
putting core set of functions in place, software companies will become
more like consulting firms and consulting firms will become more like
software companies, not duplicating the same basic functions over and
over, but concentrating their efforts on the the work that lies over
the interest horizon.



Write clay@shirky.com with questions or comments.

Send this essay to a friend:

Enter the email address of the recipient. Multiple addresses need to be separated by commas.

Your name:optional

Note: Your name, and your recipient's email address, will only be used to transfer this article,
and will not be stored or used for any other purpose.

Add optional message:

Send the article URL only
Send the article as HTML
Send the article as plain text

www.shirky.com Home

shirky.com Clay Shirky's Writings About the Internet
Economics and Culture, Media and Community, Open Source