Fat Tails: Sometimes the average is anything but average

Brian Hayes in American Scientist:

Screenhunter_10_apr_17_1819You’ve probably heard of Lake Wobegon, the little town in Minnesota where all the children are above average. There’s been much head-scratching about this statistical miracle. What happens to the kids who fail to surpass themselves? Are they shipped across the lake to another little town, where all the children are below average? That practice wouldn’t necessarily work to the detriment of either community. It might be like the migration from Oklahoma to California during the Dust Bowl years, which Will Rogers said raised the average intelligence of both states.

One small town that beats the law of averages is strange enough, but even more mystifying is the finding that Lake Wobegon is not unique—that in fact everyone is above average. In 1987 John J. Cannell, a West Virginia physician and activist, discovered that all 50 states report that their children do better than the national average on standardized tests. (And this was years before the No Child Left Behind Act!)

I can’t promise to resolve these paradoxes. On the contrary, I’m going to make matters worse by describing still more funny business in the world of averages. The story that follows is about a data distribution that simply has no average. Given any finite sample drawn from the distribution, you are welcome to apply the usual algorithm for the arithmetic mean—add up the values and divide by the size of the sample—but the result won’t mean much. Whatever average you calculate in this way, you can improve it just by taking a bigger sample. Perhaps this is the secret of the Lake Wobegon school board.

More here.