by Malcolm Murray
The Paris AI Summit the other week might have been the end of a 10-year run for AI safety as Azeem Azhar, the creator of Exponential View, put it. The concept of AI safety, which can be said to have started in earnest with Nick Bostrom’s 2014 book Superintelligence, had a 10-year run, in which it grew in understanding and acceptance among the public and decisionmakers. Subsequent books, like Max Tegmark’s Life 3.0 and Stuart Russell’s Human Compatible, further established the field. High-level AI safety principles were declared in Asilomar in 2017, at that time by all notable AI scientists and leaders.
Further, this was already of course before AI was even actually any good at anything but very narrow tasks. When AI capabilities actually caught up with the hypothetical concerns, with GPT-3 in 2020 and came into the eye of the public in a broad way with ChatGPT in 2022, it led to AI senate hearings, the UK Bletchley Park summit, the Biden Executive Order, the voluntary commitments on the AI labs, the attempted bill SB-1047 in California and the addition of general-purpose models to the EU AI Act.
That 10-year run now seems to be largely over, or at least severely weakened. We had already seen the Trump administration repealing Biden’s Executive Order on AI, removing the ability for the US government to test AI developers’ models for safety. Then, at the Paris AI Action Summit, it became abundantly clear that the world has turned away from AI safety. The main summit had banners stating “Science, not Science Fiction”, the speech from J.D. Vance was very clear that the focus would be only on AI opportunities, Macron focused on investments – the announcement of a new French data center – not mentioning any downsides (“plug, baby, plug”). The voluntary AI developer commitments – the Frontier Safety Frameworks – that had been a focus of earlier UK and Korea summits were glossed over completely. Perhaps most significantly, the formidable State of the Science report led by Yoshua Bengio, the IPCC-style report which was commissioned at the earlier summits and completed for this one, was not mentioned at all in the main event. This report, which was meant to establish a common basis for discussion, was in fact relegated to a side event the week before, at a university two hours outside of Paris.
This should perhaps not have come as a great surprise. With all major technological advances of the past, it is typically only after the public sees the dangers firsthand that safety becomes a concern and regulation is put into place. That pattern was repeated over and over, from automobiles to aviation to nuclear power. AI safety was perhaps an anomaly in the amount of interest it actually did raise already while only being a theoretical concern pre-ChatGPT.
The societal hedonistic treadmill effect that seems to be at play, where the public has ever-shortening windows of amazement, doesn’t help create any urgency. The arrival of TV created amazement for years. With smartphones, it took maybe a few years for people to cease to be amazed that all the world’s knowledge was in our pockets. But with AI, it seems all we get are a few tweets. This era of AI gave us the ability to create completely realistic artificial images in seconds that we can’t tell apart from human-created ones, or write code that is better than all but a few hundred humans. But after a few tweets, it seems the public then goes “thank you, I’ll take that, what else you got”? The exponential curve looks flat when you’re standing on it. Technology is magic until it suddenly works and then quickly utterly mundane. We blew through the Turing test a while back and no one even noticed or cared.
At the same time as the AI safety wave crested and fell, of course, the technological progress has continued unabated. When it seemed like progress was plateauing along the scaling law of training compute that powered the advances of the past years, a new scaling law quickly opened up in terms of inference compute, leading to continued exponential progress and saturation of benchmarks created only months earlier. On the fittingly-named Humanity’s Last Exam, o3 scores are double that of o1 from a few months earlier, and more than double those of GPT-4. Google DeepMind just released an “AI co-scientist”.
So where does that leave us now? If we think about the possible worlds awaiting us, the one I would ascribe the highest probability is one where AI will need to have a Chernobyl moment before any significant checks and balances are put into place. The hope in this world is that we do get early warning shots that are convincing to people and policymakers without actually being too harmful.
A more optimistic world that could await us is one in which the EU AI Act’s Code of Practice continues on its current path and does not get watered down too much. If it keeps striking the right balance between putting in place a few, size-appropriate compliance requirements, while still leaving plenty of room for innovation, as I think the current draft is doing, there is some probability that we could see a decent Brussels effect, as has been the case with other EU legislation such as GDPR. Korea’s AI regulation in December was influenced by the EU AI Act and a bill in Brazil likewise. The UK will likely eventually introduce its AI bill also. If these could be coupled with progress in regulation on the state level in the US, we may potentially continue to make good progress on reining in AI excesses. Especially if we get some regulation that specifically addresses uncertainties around liability .
Another quite high-probability world is one in which the current trajectory is locked in for a few years, at least until “AGI” (using the definition of the ability to complete any human-level cognitive task), without any meaningful regulation coming, but that we could then see controls being put in place for anything above AGI. There is huge build-up in compute, synthetic data seems to work well and new scaling laws get discovered, so there doesn’t seem to be any part of the AI triad not holding. So over the next few years, we should likely take the labs on their word that AGI is in reach. However, anything beyond AGI, i.e. ASI, is of course in a different ballpark. One might therefore hope that the most powerful world leaders – Trump, Xi, Putin – recognize the enormous advantages that ASI would yield and decide to exert control, within and between their countries.
Regardless of which of these three worlds or others we think are most likely, what is clear is that on the whole, it seems the proportion of worlds where things go well has shrunk remarkably in a short period of time, at least for now.
***
Enjoying the content on 3QD? Help keep us going by donating now.