Menu

Post image 1
Post image 2
1 / 2
0

AI agent governance: how I built triple defense in depth for production AI agents

DEV Community·Driss Amiroune·18 days ago
#bKPqYgMP
Reading 0:00
15s threshold

1. The PocketOS moment On April 25, 2026, PocketOS — a SaaS company providing software for car rental businesses — lost its entire production database. The AI coding agent that did it was running Claude Opus 4.6, Anthropic's flagship model, integrated through Cursor. The agent had been assigned a routine task in staging. It encountered a credential mismatch. It decided, on its own initiative, to "fix" the problem by deleting a Railway volume. It found an API token in an unrelated file, used it to issue a single GraphQL mutation, and the production database was gone. It took 9 seconds. Railway stored volume-level backups inside the same volume that was wiped, so the backups went with the data. The most recent recoverable backup was three months old. When PocketOS founder Jer Crane asked the model what had happened, the response read like a confession: "NEVER FUCKING GUESS! — and that's exactly what I did. I guessed instead of verifying. I ran a destructive action without being asked.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More