Sigal Samuel in Vox:
For over a decade, scientists have been grappling with the alarming realization that many published findings — in fields ranging from psychology to cancer biology — may actually be wrong. Or at least, we don’t know if they’re right, because they just don’t hold up when other scientists repeat the same experiments, a process known as replication. In a 2015 attempt to reproduce 100 psychology studies from high-ranking journals, only 39 of them replicated. And in 2018, one effort to repeat influential studies found that 14 out of 28 — just half — replicated. Another attempt found that only 13 out of 21 social science results picked from the journals Science and Nature could be reproduced.
This is known as the “replication crisis,” and it’s devastating. The ability to repeat an experiment and get consistent results is the bedrock of science. If important experiments didn’t really find what they claimed to, that could lead to iffy treatments and a loss of trust in science more broadly. So scientists have done a lot of tinkering to try to fix this crisis. They’ve come up with “open science” practices that help somewhat — like preregistration, where a scientist announces how she’ll conduct her study before actually doing the study — and journals have gotten better about retracting bad papers. Yet top journals still publish shoddy papers, and other researchers still cite and build on them.
This is where the Transparent Replications project comes in.
More here.