The Scientific Method Has Left the Building

by Muhammad Aurangzeb Ahmad

An Experiment on a Bird in the Air Pump by Joseph Wright of Derby (1768)

I recall reading a short piece of fiction as a high school student at the turn of the millenium called Catching Crumbs from the Table. It was written by Ted Chiang and later republished under the title The Evolution of Human Science. Chiang imagined a future in which genetically enhanced metahumans have superintelligence and all of science and inventing new things for the benefit of rest of humanity. However, ordinary human scientists can no longer participate meaningfully at the frontiers of research. Science continues to advance faster than ever but most of humanity can no easily understand what is going on. Howeverm Human scientists do not completely disappear. Instead, they become interpreters. Some try to understand simplified accounts of metahuman discoveries. Others test conclusions whose theoretical basis lies beyond their own abilities. Still others study the artifacts produced by metahuman science and work backward, hoping to infer the principles that made them possible. For all practical purposes science for ordinary humans has become less about discovering nature directly than about trying to understand discoveries made by minds more capable than most of humanity.

After more than two decades, I was recently reminded of this story when I came across a couple of articles in Nature. The paper described two systems that occupy a curious new territory between scientific instrument and scientific collaborator. The paper in question describes Google’s Co-Scientist which uses multiple AI agents to generate hypotheses, criticize one another’s proposals, rank them, and refine the more promising ones. Some of the hypotheses generated by the system were subsequently tested in biomedical settings. Another system, Robin searches the literature, proposes hypotheses, suggests experiments, analyzes experimental results. It then revises its ideas in light of what those experiments show. Human researchers still perform much of the physical work. It is important to note that it would be premature to describe either system as an autonomous scientist. That said, these systems are doing something qualitatively different from what we normally think scientific software is supposed to do. Read more »