I’ve been on somewhat of an unintended hiatus for the past few months, as I try to wrap up some projects, and deal with a few other things in my life. However, I just read something that has given me a kick in the pants. And I don’t mean that in a good way. In late December there was an article by Jonah Lehrer in the New Yorker titled “The truth wears off”. Much more suggestive was the subtitle, “Is there something wrong with the scientific method?”. The story discusses the “decline effect”: an article is published with startling results, and then subsequent work finds increasingly diminished evidence for the initial unexpected result. It’s as if there’s “cosmic habituation”, with the Universe conspiring to make a surprising result go away with time. The last paragraph sums things up:
The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.
I don’t particularly disagree with any of this. But it’s completely besides the point, and to untutored ears can be immensely misleading. The article is a perfect example of precisely the effect it seeks to describe (there must be a catchy word for this? Intellectual onomatopoeia?). The article gives a few examples of people finding interesting results, only to have them disappear on sustained scrutiny.