Tim Fist and Saif Khan, with Tao Burga, Arthur Tellis, Ben Schifman, Jonah Weinbaum, and Olivia Scharfman at Noahpinion:
In our last post, we evaluated the claims of a recent open letter by AI company employees calling for governments to “pace” frontier AI development. To summarize:
- Rapid progress towards fully automated AI R&D has empirical support, but it’s less clear how much it will accelerate AI capabilities or pose severe risks.
- Despite substantial uncertainty, we believe some preparatory policy action is warranted. This follows both from how serious the possible direct risks are and the risk that political backlash to AI-driven disruptions results in poorly-reasoned policy measures, such as broad bans on new data centers.
- If “pacing” becomes necessary, we think it should consist of two parts: first, specifying thresholds for when automated AI R&D is likely to pose severe risks; and second, if a threshold is exceeded, incentivizing AI companies to reallocate resources away from the most risky research, and towards activities that make further automation safer, or diffuse the benefits of existing AI faster.
Without preparation now, however, our preferred pacing strategy will be impossible to implement. In this post, we’ll describe how the US can concretely prepare for the further automation of AI R&D and the risks it entails.
More here.
Enjoying the content on 3QD? Help keep us going by donating now.