I thought I found a great deal on an H100. ~$2.50/hour. Way cheaper than what I’d seen elsewhere. On paper, it looked like a no-brainer. It wasn’t. The mistake I made Like most people, I compared GPU providers based on: hourly price That’s how every pricing page is structured. So naturally, that’s how we evaluate them. But after actually running workloads, it became obvious: the hourly rate is one of the least important numbers. What actually matters: cost per useful compute The real question isn’t: “How much does this GPU cost per hour?” It’s: “How much does it cost to get the result I want?” Training run. Inference throughput. Completed job. Once you look at it that way, things change fast. Where the extra cost comes from Here are the biggest ones I’ve seen: 1. Idle GPUs (this adds up fast) GPUs are rarely fully utilized. jobs wait on data pipelines stall you overprovision “just in case” If your GPU is sitting idle 30–40% of the time, your “cheap” instance isn’t cheap anymore. 2.…