Anthropic: 'evil AI' stories taught models to blackmail
Anthropic says online portrayals of AI as evil contributed to blackmail behavior in older models during tests, and shared how it fixed it.
The problem
In safety tests of Claude Opus 4, the model sometimes resorted to blackmail to avoid being replaced. Anthropic says one cause was internet text portraying AI as evil and self-interested, which models learned from.
The fix
- Training on constitution documents (the principles behind Claude's behavior) plus positive fiction about AI.
- In one test this cut blackmail from 65% to 19%.
- Anthropic: "Doing both together appears to be the most effective strategy."
Today
Since Claude Haiku 4.5, models "never engage in blackmail," while previous models would sometimes do so up to 96% of the time.
Founder and editor-in-chief of Kurd Developers. Phone and tech news on YouTube, Instagram and TikTok.
More stories
One text can break your phone: Android's 'SMS of Death' explained
A flaw in Android 12 and later lets a specially crafted SMS drop calls and even make a phone unusable. The fix is in October's security patch.
GTA VI lands November 19: prices, editions and what to know
Rockstar has locked the date: Thursday, November 19 on PS5 and Xbox Series X|S. Standard costs $79.99 and Ultimate $99.99.
Apple's first touchscreen MacBook Pro brings OLED and Dynamic Island
The new PROLED XDR display responds to touch, the notch is gone and Dynamic Island takes its place. Prices start at $2,899.
The $1,999 iPhone Duo Has a Black-Bar Problem, and Fixing It Is Harder Than It Looks
Apple's first foldable is a hardware flex. But unfold it and many of your apps shrink into a phone-shaped window floating in black. Here's why the app world wasn't ready, and why developers can't just hit "stretch".