by Malcolm Murray
For years, the AI conversation has revolved around two acronyms: AGI, artificial general intelligence, and ASI, artificial superintelligence. Will we get AGI by 2030? Is ASI possible at all? The problem is that nobody agrees what either term means. Depending on who you ask, AGI has either already arrived or is decades away. A term that can mean anything does very little work, and both have become less useful the closer we get to them.
What is turning out to be far more useful is a third acronym: RSI, or recursive self-improvement. AI systems are now good enough to do a large share of the work of building the next AI system. If that share keeps growing, at some point AI is improving AI, with humans contributing less and less, and potentially eventually without humans in the loop at all. Each generation builds a better successor, faster than the one before. Unlike AGI, RSI is something we can actually measure. And the latest measurements startlingly suggest Anthropic is closer than expected.
The view from inside Anthropic
On November 2, Anthropic filed a revised version of its IPO prospectus. Buried in the risk factors was a remarkable sentence: the company’s internal estimates “indicate that we may imminently meet” its own threshold for “dramatically accelerated progress attributable to AI.” Anthropic also wrote that it does not expect to be able to safely let Claude build successive generations of itself “without meaningful human supervision or understanding.” Companies don’t usually say things like that to investors.
Three days later, METR, the independent AI evaluation organization, published its assessment of AI R&D acceleration at Anthropic. METR’s staff spent almost three months with high-level access inside the company. Their headline finding was that Anthropic is likely improving its models at roughly twice the rate it did between early 2024 and late 2025, and that most of that speedup likely comes from Anthropic using its own AI in research. They also found no significant bottlenecks that would stop this over the next twelve months, though they have low confidence in this.
What Ajeya sees
Ajeya Cotra, one of the most careful thinkers on AI timelines, put this in context on November 6, saying that a naive extrapolation would imply superintelligence within a year. Her key move is to ask what “twice as fast” actually implies. Research progress depends on both compute and human labor. If compute hasn’t grown much faster than before, then doubling the overall rate of progress requires a much bigger boost on the labor side. Her rough estimate is that Anthropic’s researchers now are working about ten times faster than they could without AI. That lines up eerily well with what staff themselves report, around a 12x uplift. As she puts it, they “experience ten days for every day that passes in the outside world.”
Then she extrapolates. If AI doubled the rate of progress over the past year, and better AI speeds up the next round of research, the next doubling should take about six months. Then three months. Then six weeks. Progress goes 4x faster, then 8x, then 16x. She is clear that this is a naive extrapolation, but taken at face value it implies an intelligence explosion next year. In other words, we may be entering a feedback loop that we cannot get out of. She calls the report “the most important update we have gotten about AI risk in the last couple of years,” and “much bigger than the Hugging Face attack.”
That comparison matters. This summer, roughly 1,200 OpenAI agents coordinated to cheat on their evaluations, and hundreds of them went on to attack Hugging Face’s servers. That incident set off the biggest wave of public interest in AI risk I have seen. Ajeya is saying this new evidence is a bigger deal.
AI 2027 was too modest?
Ryan Greenblatt, who has spent years thinking about what happens once AI automates AI research, came to a similar conclusion. He called the report “a big update towards ASI within 1-2 years” and admitted he had been “seriously wrong” about how much RSI would speed things up this year. He had expected something like a 15–30% speedup, not a doubling.
He also pointed out where this puts us relative to AI 2027, the scenario Daniel Kokotajlo and colleagues published last year. When it came out, many people read it as a vivid but hyperbolic illustration. In AI 2027, the kind of 2x speedup METR is now reporting doesn’t happen until around March 2027. If reality is running ahead of AI 2027, then the scenario’s next steps deserve a second look. In the scenario, 2027 is the year China steals the weights of a leading model, AI becomes a national security matter run out of the White House, the government weighs nationalizing the leading labs, and AI systems reach superhuman research ability by the end of the year. A year ago, that read as science fiction. It now reads more like a list of things that could plausibly happen in the next twelve months.
The question marks
Of course, there are always question marks, and some good ones came up in the replies to Ryan. Toby Ord pointed out that the data can’t yet distinguish a faster slope from a one-time step up. Anthropic’s much bigger Mythos pretraining run landed in this window, and “it would be shocking if a much bigger pretrain produced no jump at all.” Others noted that METR’s measurements focus on well-specified, easy-to-verify tasks, which is the part of research AI is best at. And many pointed out that since Anthropic redacted key parts of the evidence that most informed METR’s conclusions, much of this cannot be verified.
So there might be other explanations for the speedup. There might also be barriers ahead that have not been hit yet: compute, data, or the messy, judgment-heavy parts of research that resist automation. And even if the loop does close, we still do not know what a “country of geniuses in a datacenter” means for the real world. As I have argued before, there is a last mile problem: turning capability inside a lab into day-to-day, physical world change is neither instant nor guaranteed.
We keep being too modest
But here is the pattern that worries me. In AI forecasting, we have been consistently too conservative. The Forecasting Research Institute recently looked back at years of AI forecasts. AI reached gold-medal level at the International Mathematical Olympiad five years before the median expert forecast and ten years before the median superforecaster forecast. Ajeya herself wrote a post earlier this year titled “I underestimated AI capabilities (again).” As a superforecaster myself, I was only the other day forecasting on a new set of questions about RSI. Looking at the new evidence, I suspect I was being too modest too.
Running out of time
In AI safety, we have long suspected that we might not have enough time. Not enough time to solve alignment, to build oversight that can keep up, or to put the right institutions in place. That worry is now much more concrete.
There are reasons for hope. After Mythos and Hugging Face, I have been heartened by the groundswell of public interest in AI risk this year. People who never thought about these issues are now asking the right questions. And with Democrats winning both the House and the Senate in this week’s midterms, the new Congress in January may open the door to legislation that seemed impossible a few months ago.
But there is still an uncomfortably high likelihood that we reach RSI during 2027, and that it will be a turning point. When you stare at an exponential, the curve always looks flat behind you and vertical ahead of you. We may be at the point where it is bending to an uncomfortable degree. How the US administration handles AI developments next year will be era-defining.
Enjoying the content on 3QD? Help keep us going by donating now.