Menu

Post image 1
Post image 2
1 / 2
0

CPU Humbled Me — A Kubernetes Throttling Story Hidden Between Prometheus Scrapes

DEV Community·Satyaki·17 days ago
#8yxDR1kw
Reading 0:00
15s threshold

Memory is easy. CPU humbled me. With memory, the rule is brutal but clear — cross the limit, the pod gets OOMKilled. Done. CPU? CPU is sneaky. And I ignored it for the longest time… until it broke production. Here's what happened 👇 We had an app running peacefully in-house. Then it went client-facing. Traffic surged, and suddenly ~15% of requests started timing out — most of them on DB calls. I opened Grafana expecting a smoking gun. Nothing. CPU usage looked "fine." No throttling alerts screaming at me. Just confused timeouts. The trap? Throttling happens in milliseconds. Prometheus scrapes every 15 seconds. Every bit of evidence was hiding between the scrapes.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More