Why Evals Matter Training a model is only half the job. Without a systematic way to measure what it can actually do, you are flying blind. The problem is that evaluation is easy to do badly, you can run a benchmark, get a number, and walk away thinking you know something when you don't. This post is about doing it properly on a budget. I ran three standard benchmarks against Qwen2.5-0.5B on a free Colab T4, logged wall-clock time and dollar cost for each task, and documented every methodological decision along the way. Total spend: $0.1185 . The Benchmarks I picked three tasks that cover meaningfully different capabilities rather than variations of the same thing. GSM8K (Cobbe et al., 2021) tests grade-school math reasoning. The model has to produce a chain-of-thought and arrive at a final numeric answer. Scoring is exact match, either the answer is right or it isn't. This is a generative task, which makes it slower and more expensive than the others. I used 5-shot prompting following the original paper.…