Jocelyn Kaiser in Science:
The first results of a high-profile effort to replicate influential papers in cancer biology are roiling the biomedical community. Of the five studies the project has tackled so far, some involving experimental treatments already in clinical trials, only two could be repeated; one could not, and technical problems stymied the remaining two replication efforts. Some scientists say these early findings from the Reproducibility Project: Cancer Biology, which appear tomorrow in eLife, bolster concerns that too many basic biomedical studies don’t hold up in other labs. “The composite picture is, there is a reproducibility problem,” says epidemiologist John Ioannidis of Stanford University in Palo Alto, California, an adviser to the project whose attention-getting analyses have argued that biomedical research suffers from systemic flaws. But others say the results simply show that good studies can be difficult to precisely reproduce, because biological systems are so variable. “People make these flippant comments that science is not reproducible. These first five papers show there are layers of complexity here that make it hard to say that,” says Charles Sawyers, an eLife editor and cancer biologist at Memorial Sloan Kettering Cancer Center in New York City.
The cancer biology project was inspired by reports from two companies that when they tried to follow up on dozens of papers pointing to potential new drugs, they could not replicate as many as 89% of the studies. But the firms, Bayer and Amgen, did not reveal the specific papers or many details of their attempts. So in 2013 the nonprofit Center for Open Science in Charlottesville, Virginia, which had led a replication project for psychology papers, teamed up with Science Exchange of Palo Alto, a service that matches scientists with contract labs that do experiments for hire. The partners won $2 million from a foundation for a large-scale cancer replication effort.
More here.
