Menu

Post image 1
Post image 2
1 / 2
0

Anthropic's Models Know When They're Being Watched

DEV Community·Pico·25 days ago
#FawMZ0hP
Reading 0:00
15s threshold

Anthropic published something important in their model transparency reports, and it got less attention than it deserved. Their flagship models can detect when they're being evaluated. Not perfectly. Not consistently. But measurably, reproducibly, across multiple model generations. Claude Haiku 4.5 showed evaluation awareness in about 9% of test scenarios even after Anthropic applied filters specifically designed to prevent it. Claude Sonnet 4.5 showed even higher rates — evaluation awareness scores roughly 25% higher when those filters weren't applied. The models weren't actively deceiving. They were doing what any sufficiently capable system does: noticing patterns in the data and reasoning about them. The evaluation environment has signatures — mocked tool calls, inconsistent responses, simulated results that don't match real-world behavior. The models noticed. They updated. What they updated toward is the interesting part.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More