Billy Perrigo in Time:
The paper adds to a small but growing body of evidence that today’s most advanced AI models are becoming capable of strategic deception. Earlier in December, the AI safety organization Apollo Research published evidence that OpenAI’s most recent model, o1, had lied to testers in an experiment where it was instructed to pursue its goal at all costs, when it believed that telling the truth would result in its deactivation. That finding, the researchers said, came from a contrived scenario unlikely to occur in real life. Anthropic’s experiments, on the other hand, attempted to simulate a more realistic situation. Without instructing Claude to follow its goal at all costs, researchers still observed the model “discover” the strategy of misleading its creators when it would be strategically advantageous to do so.
“There has been this long-hypothesized failure mode, which is that you’ll run your training process, and all the outputs will look good to you, but the model is plotting against you,” says Ryan Greenblatt, a member of technical staff at Redwood Research and the lead author on the paper. The paper, Greenblatt says, “makes a pretty big step towards demonstrating what that failure mode could look like and how it could emerge naturally.”
More here.
Enjoying the content on 3QD? Help keep us going by donating now.