Daniel Engber at Slate.com considers the issue in light of the Journal of Reproductive Medicine study on the relationship between prayer and fertility.
“Peer review is the gold standard of modern science. For medical researchers and other scientists, it’s the gateway to funding, publication, and career advancement. When they apply for government grants from the National Institutes of Health or the National Science Foundation, their proposals are reviewed by a panel of their colleagues. When they submit their completed work for publication, journals and university presses ask for the opinions of others in the field. And when they apply for jobs or tenure, scientists are judged largely on the basis of their peer-reviewed publications.
Scientists give peer review so much authority because they view it as a part of the grand tradition of scientific inquiry—an extension, even, of the formal experimental method. Peer evaluation is the endpoint of a cautious progression from theories and predictions to experiments and results. . . [S]cientists have claimed that peer review filters out lousy papers, faulty experiments, and irrelevant findings. They say it improves the quality of an accepted paper by providing helpful comments for revision. And they can’t imagine a better way to accomplish these goals.
So, what explains the Columbia prayer study? Journal editors will tell you that peer review is not designed to detect fraud—clever misinformation will sail right through no matter how scrupulous the reviews. But the prayer study wasn’t a clever fraud.”