by Muhammad Aurangzeb Ahmad
I recall reading a short piece of fiction as a high school student at the turn of the millenium called Catching Crumbs from the Table. It was written by Ted Chiang and later republished under the title The Evolution of Human Science. Chiang imagined a future in which genetically enhanced metahumans have superintelligence and all of science and inventing new things for the benefit of rest of humanity. However, ordinary human scientists can no longer participate meaningfully at the frontiers of research. Science continues to advance faster than ever but most of humanity can no easily understand what is going on. Howeverm Human scientists do not completely disappear. Instead, they become interpreters. Some try to understand simplified accounts of metahuman discoveries. Others test conclusions whose theoretical basis lies beyond their own abilities. Still others study the artifacts produced by metahuman science and work backward, hoping to infer the principles that made them possible. For all practical purposes science for ordinary humans has become less about discovering nature directly than about trying to understand discoveries made by minds more capable than most of humanity.
After more than two decades, I was recently reminded of this story when I came across a couple of articles in Nature. The paper described two systems that occupy a curious new territory between scientific instrument and scientific collaborator. The paper in question describes Google’s Co-Scientist which uses multiple AI agents to generate hypotheses, criticize one another’s proposals, rank them, and refine the more promising ones. Some of the hypotheses generated by the system were subsequently tested in biomedical settings. Another system, Robin searches the literature, proposes hypotheses, suggests experiments, analyzes experimental results. It then revises its ideas in light of what those experiments show. Human researchers still perform much of the physical work. It is important to note that it would be premature to describe either system as an autonomous scientist. That said, these systems are doing something qualitatively different from what we normally think scientific software is supposed to do.
For most of the modern computing era, machines occupied a fairly clear place within science. The original promise of computers was that they were able to calculate much faster than humans and store and access vastly more reservoirs of information. With simulation modeling, we were also able to simulate complicated systems and and detect patterns That are too complicated for humans to detect in the first place. With the widespread adoption of machine learning for predictive modeling in the sciences, one could still make the argument that human scientists are deciding what problems are worth solving and how the results should be interpreted. Systems that generate hypotheses Make things more complicated. Consider a computer system that calculates the consequences of a scientific idea is still recognizably an instrument. Let’s contrast it with the system that proposes the idea itself. At this point, it’s starting to look more like a scientist because, traditionally, hypothesis formation is the purview of the human scientist.
There’s something ironic in this arrangement if you think about it. Science fiction imagined that advancements in computing and AI will one day free humans from the drudgery of physical work. If these trends continue, then we may end up with an opposite arrangement. Imagine a world where there’s a machine, an AI model that searches the literature and generates a hypothesis. Based on the hypothesis, it recommends certain experiments be done. It would then gather the data from the experiments, interpret it, and suggest conclusions. The human scientists are then left to just do the experiments. As strange as that sounds, there are even more parts of this process which could be automated. The next step would be automated laboratories, many of which already exist, but in this imagined near future, they will be tightly coupled with the hypothesis generation and experimentation process. This is not to say that scientists will disappear. However, how science is done will greatly change.
Let’s consider a scenario where there is an AI system whose task is to develop a vaccine. It searches the complete storehouse of human knowledge and all the relevant data available to it, and automates experimentation. When the additional data comes in, it generates new hypotheses and does additional experimentation. Eventually, it arrives at a candidate vaccine, which is then tested in animals. Afterwards, that is extended and tested in humans, and it seems to work. Now also imagine: what if the human scientists are not able to give a satisfactory explanation of how the system arrived at the vaccine? So now the question is: is this really science that we are doing anymore? This scenario is actually not as strange as it seems. Consider how big research is done even now. For many large projects, it’s safe to say that no individual person understands it in its entirety. Think about climate models and particle physics simulations. The complexity of these models far exceeds the grasp of any single scientist. Also, consider a physician that prescribes a drug. It would be safe to say that, in most cases, the physician does not really understand the manufacturing process, and the statistical analysis and molecular interactions That makes the drug feasible. .
Modern science works because understanding is distributed. One person understands the assay, another understands the statistical method, and another knows the instrumentation. Scientific institutions allow us to rely on this division of labor. Peer review, replication, professional specialization, and scientific reputation all help create trust between people who cannot independently verify everything they use. The potential risk that we may be seeing here is that there may be parts of this process which cease to be understandable to humans at all. Thus, a machine could arrive at a reliable result through an inferential path involving an enormous number of interacting variables, simulations, model calls, searches, and intermediate representations. We might be able to test the output without being able to reconstruct the route that produced it. It may be tempting. to treat this as a philosophical luxury. If a drug works, does it really matter whether anybody understands the route by which it was discovered? If a model predicts a hurricane accurately, the people in its path would presumably prefer a correct warning to an elegant explanation. Engineering has always tolerated a certain amount of pragmatism, and scientists themselves frequently use tools whose inner workings they do not fully understand.
Modern Science has historically offered not only successful prediction but its great achievements have often been acts of compression. This has often involved turning a bewildering variety of observations into ideas that human beings could carry around in their heads. Think about Newton connected the falling of an apple with the motion of the Moon. Darwin provided a mechanism by which the apparent design of living things could arise Through the trial-and-error process of evolution . Germ theory reorganized an enormous range of diseases around the idea of microorganisms. That kind of understanding has practical value as well. Explanations travel. They allow scientists to notice analogies between fields, to see when an idea might apply somewhere unexpected. One way to think about it is that a prediction might tell us what will happen given certain data. An explanation, on the other hand, gives us something to think with. I think this is where Chiang’s story seems especially perceptive. He did not imagine that ordinary scientists would simply become useless Rather, the task of the human scientist now becomes to make things intelligible again.
The scientist in such a world would not merely be a diminished version of the scientist we know today. The role might become, in part, one of epistemic custodianship: preserving the connection between discovery and understanding. There is also an additional task that scientific automation does not make disappear, which is deciding what is worth knowing. Nature contains an effectively unlimited number of true facts. Science has never been the indiscriminate collection of all of them. We investigate certain questions because people have diseases, because particular phenomena puzzle us, because some problems appear beautiful etc. A sufficiently powerful machine could be extraordinarily productive while pursuing questions that human beings find utterly uninteresting. Scientific importance is not a property that can always be read directly from the natural world. It reflects human priorities, needs, aesthetics, and values. Even a system vastly better than us at solving scientific problems would still require some account of which problems ought to matter.
There’s another way to think about this scenario: For most of the history of science, obtaining knowledge from nature was the difficult part. Ted Chiang imagined a world in which knowledge had become plentiful and understanding had become the bottleneck. There is still a great distance between current AI co-scientist systems and Chiang’s metahumans. Today’s AI agents are brittle, error-prone, dependent on human knowledge, and nowhere close to constituting a parallel civilization of scientific intellects. It would be easy to mistake an interesting early development for something resembling what Ted Chiang describes. That said, Chiang’s story feels newly useful because it directs our attention away from the usual question of whether machines will replace scientists. A stranger possibility is that science continues as it is, the human scientists will remain employed. However, an increasing fraction of discovery takes place somewhere beyond the frontier of ordinary human comprehension. Suppose if that happens then the task of the scientists will not simply be to keep making discoveries but it will be to keep discovery connected to understanding.