Introduction We hit a scaling cliff when our product moved from a few thousand concurrent users to tens of thousands. The thing that looked trivial in staging — pushing events over WebSockets and orchestrating AI agents — started manifesting as tail latency spikes, connection storms, and a surprising amount of bookkeeping code in our app layer. Here’s what we learned the hard way building a realtime, event-driven backend for AI workflows and multi-tenant SaaS. The Trigger The immediate trigger was simple: a big customer started running thousands of long-running inference sessions using multiple agents that exchanged messages in realtime. At first, this looked fine — we had a single message broker and a WebSocket cluster. Then: Connection count grew beyond our sticky routing assumptions and we saw frequent disconnects. Message ordering guarantees we relied on became inconsistent under retries. Orchestration state (who’s waiting on which agent) lived in app memory and was lost on restarts.…