Of Cathedrals and Bazaars

Cathedrals are awe-inspiring creations. Many are graceful and beautiful, but even a graceless, cobbled-together cathedral took many years of hard, organized work. A bazaar, in contrast, can be started in a vacant lot one afternoon and knocked down the same evening when the customers have decided to go home.

On the other hand, a truly successful bazaar is almost always the result of years of community involvement. Merchants and customers both have to make room in their schedules for the bazaar. The "vacant lot" must be kept unoccupied, lest there be no place for the bazaar to meet.

Bazaars also have a degree of flexibility that cathedrals cannot possibly emulate. If a fruit stand needs more room, it will find a way to expand or move to a larger venue. Cathedrals, in contrast, are literally "cast in stone"; adjusting the room size is not a task to be taken lightly!

A couple of months ago, Vicki Brown and I discussed "The Cathedral and the Bazaar", a paper that Eric Raymond presented at this Summer's Perl Conference. Rather than send you scurrying for an old issue, here is what we said about it:

The Cathedral and the Bazaar

Eric Raymond is best known as the editor of "The New Hacker's Dictionary" (MIT Press, 3rd edition, 1996, ISBN 0-262-18178-9). His talk on "The Cathedral and the Bazaar" is a fascinating look at the ways (free) software is developed. Eric has given this talk in several venues and has even created a web page for it (www.ccil.org/~esr/writings/cathedral.html).

The cathedral model attempts to issue "perfect" releases, using a small set of highly-skilled developers and relatively long release cycles. This model allows users to have a relatively high degree of confidence in the issued code, at the expense of a somewhat slow development cycle. In the cathedral model, users must never see bugs. Many freeware projects use the cathedral model; it is almost universal in commercial software development.

The bazaar model, in contrast, issues new releases whenever significant changes have been made. No claim is made for perfection, but releases come very frequently and incorporate bug fixes very promptly. (To shield naive users from the perils of rapidly evolving code, most bazaar-based projects have release schemes that provide both debugging and production snapshots.)

Confounding Eric's preconceptions and much of the available literature, the bazaar model has been shown to work for substantial software projects (i.e., Linux). What's more, the code tends to develop (and become bulletproof) at a very rapid rate. Eric has analyzed the reasons for this and performed a small-scale experiment to test his theories. I strongly recommend that you take a look at his web page.

A Bazaar Approach to Publishing

Prime Time Freeware (PTF) is heavily involved these days in creating, publishing, and promoting a book on MacPerl. We are trying to produce a high-quality product in a rather short period of time. Most publishers would take a year to do this project; we're allocating six months and hoping for less!

We are also working on building up the MacPerl community. Part of our reason is altruistic: we actually believe that MacPerl fills an important technical niche and that many Perl and Macintosh users would benefit from learning about its existence.

On the other hand, we also realize that an active MacPerl community could be very useful to us in getting out the word on our product. (Being the only fisherman at an empty fishing hole is not all that useful...) So, we are (ahem) strongly motivated to assist MacPerl in reaching its full potential.

In pursuit of this goal, we have created the MacPerl Pages (www.ptf.com/macperl), sent numerous announcements to email lists and media contacts, had meetings with marketing folks at Apple, etc. In short, the usual guerilla marketing blitz...

At the same time, of course, we are writing and editing like crazy. At some point in this craziness, yours truly was struck by a thought: Why not apply Eric's Bazaar model to the editing process? By bringing assorted volunteers (and near-volunteers :-) into the process, we can establish checkpoints, get useful feedback, and help to build a receptive audience for the finished product.

The results aren't all in yet, but the early returns are very promising. Several weeks into the book's development, we have more than three dozen reviewers, ranging from casual to intense. With each new snapshot, we pick up a few more critics; I expect the grand total to be well over a hundred!

The comments we have received also span a wide range. Some are very nit-picky, dealing with typos, arithmetic errors, etc. Because, as Eric notes, finding bugs is much harder than fixing them, these folks are real jewels! Other reviewers have discussed our organization, pedagogical style, and degree of Political Correctness.

The Economics of the Bazaar

One of our reviewers asked how we found the time to edit chapters, work on web pages, etc. A good question: it does take time to create new "product" and keep the "stall" in order. On the other hand, we could not possibly afford to pay for the kind of feedback we've received, and we're getting it all for free!

Fundamentally, this is just the "potlatch" economics of the Bazaar at work. PTF is giving MacPerl a communal watering hole (the MacPerl Pages) and some much-needed publicity. This will bring in new users and programmers, which will help everyone in the community.

As a Bazaar grows, the number and variety of available resources increases. In the case of MacPerl, this means that there will be more volunteers to answer questions, find bugs, contribute explanations to the FAQ, etc.

By writing a book on MacPerl, we are organizing and disseminating information on MacPerl, a Good Thing To Do. Our reviewers respond to this "donation" by making their own. They sit down with our efforts, look them over, learn what they can, and let us know what's wrong.

PTF, in turn, pays for this volunteer effort by giving public acknowledgements and credit. Not only will the book have a substantial acknowledgements section, the MacPerl Pages have a page dedicated to listing folks who have helped out with the project. Finally, we will give away some tangible goodies to volunteers who have been really helpful.

There are some unexpected psychological side-effects to using the Bazaar model. The main ones have to do with the creator's ego involvement. The problem is not, as you might expect, that creators are unwilling to give away credit. As Eric notes, any substantial project will have plenty of credit to go around.

Rather, the problems lie in accepting criticism and being willing to "let go" of a draft whan it is still known to have problems. It's hard enough to deal with nit-picky criticism from a single editor; are you really ready to encourage input from dozens?

And, at two in the morning, are you secure enough in your convictions to push out something that you know can be improved with only a few hours of work? I have gone to bed at five am, trying to get out a "clean" release, so I understand this issue all too well!

Upcoming Bazaars

I expect to try the Bazaar model in future projects, when it seems appropriate. If the reviewing (i.e., debugging) effort looks distributable, and the schedule allows me to distribute intermediate results, there is very little reason not to try the approach again.

I would also encourage other who work in suitable environments to consider how multiple rounds of editing and feedback could improve the final product. I realize that many commercial environments do not allow projects to "wash their laundry in public", but the benefits can far exceed the risks.