Come Together, Right Now

Steven Johnson in Discover Magazine:

Chances are, whether you’re aware of it or not, you’ve participated in a spontaneous mass audience on the World Wide Web. Someone somewhere decides to share something that catches your interest: a home video, say, of the tsunami taking out a beach resort. At first that sort of offering gathers an audience slowly. A few people send a link to friends, but soon the links become part of a positive feedback loop, and before long big media news sites have noticed the file, and the initial cluster of visitors becomes a swarm…

There’s a catch. Buzz about a new video clip spreads in a distributed way, but the process of viewing the clip remains defiantly centralized. The millions of people who heard about a tsunami video got word of it from thousands of different sources, but they all descended on a single Web server that hosted it. And when a million people all try to request a file from a server ill-prepared for the traffic, the result is like a thousand people showing up to take a ferry designed to hold a hundred passengers: Either most of the visitors get turned away, or the ferry sinks…

For some time now, there has been an ad hoc means of dealing with logjams created by spontaneous mass audiences—mirror sites containing copies of the original file. So when someone spreads news of a hot link, they typically offer a supplementary list of mirror sites in case the original is down. The idea is to manage the swarm by dispersing it.

An even better idea has emerged recently. Instead of creating mirror sites, some Webheads create a so-called torrent when a file is in great demand. That enables others to download the file using BitTorrent, a small but elegant program that actively encourages swarm formation and has a paradoxical effect. The more popular the file, the easier it becomes to download.

More here.