by Robyn Repko Waller
We think of AI as the stuff of science, but AIs are born artists. Those artistic talents are the key to their scientific power and their limitations.
We often seem to conceive of artificial intelligence (AI) as implementing an abstract, advanced version of the scientific method. Think, for instance, of recent successes in utilizing machine learning techniques to identify potential effective in-use drugs for combating severe COVID in the elderly. Here a machine learning technique, auto encoder, analyzed large data sets of genetic expression and how these genetic expression patterns were impacted by available drugs as well as by SARS-CoV-2. With AI, the pace of clinical trials, and so the timing of life-saving treatment, is quickened.
Or take the recent application of machine learning, specifically deep residual neural nets, in astrophysics to datasets of known gravitational lenses as a method of locating previously unknown galaxies exhibiting gravitational lensing. Gravitational lenses are observable warping of spacetime in images of distant galaxies. Observation of these gravitational lenses is critical for furthering understanding of the fundamental nature of the Universe, including black matter, but are not easily detected despite powerful observational telescopes and spectroscopic technology. Such discoveries, made possible via the coordination of machine learning and other tech-driven astrophysics, have doubled the number of known gravitational lenses, significantly advancing our ability to understand the fundamental properties of spacetime.
These collaborations between machines and human scientists seem to be a good fit precisely because what the scientists aim to do — identify the underlying workings, patterns, and structures of observable phenomena of interest in our natural world. The power of human scientists, utilizing experimentation and sophisticated instrumentation, to fruitfully theorize about this underlying reality is great, but combined with the power of AI is exponentially greater. And faster.
There’s an urgency, particularly in the case of medicine, to understand now. And the AI at issue here, machine learning, excels at uncovering the hidden complexity in vast data sets that we human investigators alone cannot find. In this way, AI might be fittingly thought of as super-scientists, elucidating the structure of the unobserved world — that which is beyond the scope of human senses and our traditional scientific instrumentation.
But the AI as scientist conception runs the risk of missing out on a — the — characteristic feature of AI, particularly machine learning. Once this feature of machine learning is thrown into relief, AI as artist seems a more fitting conception.
Here we’ve been focusing on the products of machine learning approaches in science — identification of available drugs for new diseases, detection of previously unknown gravitational lenses for observation, etc. Here the scientists employing machine learning have a broader aim in mind: find new drug candidates with which to expand efficacious clinical treatment of COVID, find new data on which to test theories of dark matter and measure the Hubble constant, etc. Moreover, there is a whole host of background assumptions, theories, and findings driving this work in arenas such as immunology, human genetic sequencing, general relativity, spectroscopy, and so on. A macro-level behavior of a system to be explained in other terms. In other words, to the scientists involved, there is a big picture to be filled in. A more nuanced, fine-grained understanding of the genetic expression patterns or the properties of images of galaxies matter insofar as knowledge of these entities is fruitful in conjunction with our best scientific theories in approximating the broader aim.
But to the AI, this task is inverted. Rather than working from some intellectually or prudentially curious place on the big picture of it all, AI works from the fine-grained features of data, the initial feature space of training data, to piece together meaning. What is meaningful to the AI? Any features, either initial ones in the training set or abstracted ones, that offer better predictions for its assigned task. For instance, AI moves from the initial pixel space of a photograph, say grey-scale values of individual pixels, to higher-level features of utility for statistical analysis. The high-dimensional feature spaces includes “mashed and ripped” versions — reconfigurations — of the initial features more amenable to analysis. This ability to “make sense of” the structure and connections among features so fine-grained that they are, computationally or semantically, inaccessible to human investigators is why we employ machine learning. But such focus on these created high-dimensional feature spaces marks a difference in attention from that of its human collaborators’.
Relatedly, then, machine learning is exploratory in spirit. Whereas human scientists often work with an eye toward the testing of some theory along with the history of the best available theories, mechanics of instrumentation, and established experimental protocol on their backs, machine learning algorithms approach the training set unaware of this broader theoretical, sociological, and historical framework. Rather, the machine learning algorithm explores the initial feature space.
Of course, that is not to say that machine learning is isolated from the larger sociopolitical world. Training sets represent, explicitly or implicitly, aspects of our world, and when that training set captures socially sensitive data, such as race, gender, or socioeconomic class, our societal social biases can be unwittingly captured in that data. Hence, the exploratory nature of machine learning does not recommend an uncritical trust of its output. Nonetheless, its activity is (typically) fundamentally exploratory.
Now this focus on fine-grained representation and exploratory nature seems more familiar to that of a human enterprise often contrasted with the sciences, that of the arts. Much of visual art involves centrally fine-grained representation of the world. The product is a high-dimensional space ripe for exploration by both artist and viewer alike. But the similarities of machine learning algorithms and artists as creators of representation go deeper than this.
Consider abstract art. We’ve noted that machine learning, in aiming to make an accurate prediction, constructs a high-dimensional feature space that contorts and reconfigures the data representing features of the natural world. Likewise, artists “mash and rip” features of visual reality to create a system of forms, texture, and color that richly represent features of visual experience in an alien way. The subject of representation is abstracted but represented none-the-less.
Take, for instance, Claude Monet’s The Houses of Parliament (Effect of Fog) (1903-4). The River Thames, the iconic neighboring Houses of Parliament, and the human inhabitants of London are depicted at dusk, in a representational detail yet blurriness that is befitting of a London pea souper albeit not one of this world. When one confronts a piece of abstract art, one’s first impression isn’t to always jump to the “big picture” of it all, but rather to take in the representationally rich details of the subject matter. The broader picture emerges, if it does at all, via exploration of the finer details. The ‘local’ fogginess and duskiness of the scene gives way to the wider perspective of the Thames.
If AI is artist, we shouldn’t be surprised that, indeed, AI excels at abstract art. And poetry. Check out this gallery in Chelsea of AI-produced portraits, haunting depictions of depth and abstraction of human forms. Or these dark, metaphorical poems a machine learning algorithm produced when trained on the Twitterverse as a dataset. (Interestingly, instances of AI can perform impressively well on image generation as well as language generation more generally (think GPT-3), although such attempts are still subject to notable limitations.)
What does the AI as artist conception mean for machine learning approaches in science? Perhaps surprisingly, AI, qua artist, powers our scientific discoveries.