by Malcolm Murray
AI safety has what I would call a “last mile” problem. This summer, AI safety suddenly and very unexpectedly went mainstream. Whether it was the appearance of Jacob Coxon on TV shows or other former lab employees on the cover of New York magazine, the news of the OpenAI agents spontaneously forming a bizarre collective during their attack on Hugging Face or dry tinder left over from the Mythos moment earlier in the year, the existential risks of AI had broken through.
For those of us who have been working on these risks for the past 3-4 years, this was quite a surprise. The field has obviously tried to draw attention to its cause for some time, but it turns out that it’s not letters or statements from Turing Award and Nobel Prize winners or books with “Everyone Dies” in the title that move the needle. Instead, it takes a combination of resignations, hacks of foreign government websites and salient real-life incidents.
It is, of course, very positive that AI safety, including its existential aspects, has entered public discourse. We are finally seeing a much overdue flurry of legislative activity in the US. However, there is still a “last mile” problem. By this, I mean that while there is growing recognition that AI could be dangerous to humanity, the average person still doesn’t understand how it would kill us all. I receive questions such as: “But if I’m in my living room, how will AI get to me?” or “Why does me asking AI about my recipes mean that it will want to kill me?”
The term comes from infrastructure fields such as telecom or energy, where it refers to the final stretch: actually delivering the service to individual households. How does AI go from a “country of geniuses in a datacenter” that may have malicious intentions to actually killing all humans, or, for that matter, even having an impact on the “real”, non-digital world? It is simply not intuitive.
I would argue that the AI safety field has failed to bridge this gap, to show people concretely how adverse outcomes might actually manifest. Let’s look at some of the most discussed ways in which this could happen, as well as some of the common objections.
The pathways that have been most discussed are bio, cyber, gradual disempowerment, nuclear, gradual environmental degradation, nano, hired humans and “unknown but inevitable”. In their simplest form, they are as follows:
- Bio means the AI designs, weaponizes and releases a highly potent new pathogen that combines the mortality rate of smallpox with the transmissibility of COVID-19.
- Cyber means the AI conducts cyberattacks that cripple critical national infrastructure, such as electricity or water.
- Gradual disempowerment means humanity gradually hands over the reins to AI, without recognizing it is doing so, thereby losing all its power and influence.
- Nuclear means the AI infiltrates military systems and launches nuclear strikes.
- Gradual environmental degradation means AIs run more and more of the economy, building solar panels and data centers over an increasing proportion of available land, eventually causing humans to lose access to food and oxygen.
- Nano means the AI invents and releases self-replicating nanomachines that act like bacteria or that use up all of Earth’s materials.
- Hired humans means the AI pays or persuades humans to take actions that are detrimental to other humans.
- Unknown but inevitable is a catch-all for arguments that we cannot fathom what pathways an entity more intelligent than us would pick. We are the Aztecs facing the Spaniards’ guns.
All of these have issues, and none of them are anywhere near bulletproof. Skeptics point out that designing a pathogen is not the same as manufacturing and dispersing one, that nuclear command systems are air-gapped, and that intelligence does not automatically confer power in the physical world. It is therefore not surprising that the last mile problem exists.
A promising way forward would be to go through the classes of scenarios and make them better defined and more precise. First, some of these scenarios suffer from avoidable definitional own goals. For example, most of them are unlikely to lead to the death of all humans, down to the very last one. In almost all scenarios, except a true “grey goo” scenario in which nanomachines self-replicate without end, some humans are likely to survive. However, this seems like the wrong definition to use. I think most people other than Parfit would agree that an event that kills 7 billion people, even if 1 billion survive, is a catastrophe too horrible to contemplate. So the bar should not be “kill all humanity”. After all, the bar for catastrophic risk in California’s recent frontier AI law is “only” 50 deaths or serious injuries, some eight orders of magnitude below all of humanity. Dropping the “kill all humanity” bar and defining catastrophic risk somewhere between those two could help the debate.
Second, some of the arguments are logically solid but simply require too much imagination for the average person to be swayed by them. Setting these aside would remove the “unknown but inevitable” argument, which few people can in practice take in, and the nanotechnology scenarios, which remain entirely unproven. Similarly, gradual disempowerment describes a process that is likely too nuanced. Humanity has always become reliant on every new technology it invents, so it is very hard to draw the line at just how much disempowerment is too much. Third, scenarios where AI pays or persuades humans are not a complete path to extinction on their own, rather an enabler that should figure as a part of many of the other scenarios.
Pruning the scenarios in this way would leave us with scenarios that are a bit more grounded in reality, such as bio, cyber, nuclear and gradual environmental degradation that could then be further developed. For each, we have already seen how it affects humanity in smaller instances (COVID, Ukraine power grid hacks, Hiroshima, loss of arable land in many regions of the world due to drought), and the leap of imagination is reduced to one of scale.
What is needed for these scenarios is to flesh out in much more detail exactly how individual humans would be affected and how the countermeasures that people believe to be in place could be overcome. A combination of describing historical near-misses and showing AI’s increasingly strong persuasive powers could be useful here. Looking at the simple economics of how AI-driven corporations would work could also be helpful.
I think that this is the key task ahead in AI safety. A window of opportunity has opened for managing the risks of AI better, now that the public is paying attention, but there is still a long way to go before that attention turns into action.
***
Enjoying the content on 3QD? Help keep us going by donating now.