In Week 11, Tenacious-Bench reported: Delta A = -2.34 pts, 95% CI [-11.09, +6.20], p = 0.71 (not significant) Delta B = +22.18 pts, 95% CI [+14.43, +29.82], p = 0.0 (reported significant) At first glance, this looks straightforward: one result is meaningful, one is not. But this interpretation can be wrong if the benchmark is underpowered for the effect sizes we actually care about. This post answers two practical questions: With 216 binary pass/fail tasks, what size improvement can this benchmark reliably detect at 80% power? Is reporting p = 0.0 valid when bootstrapping with 2,000 samples? 1) The key statistical gap: significance without power is incomplete A p-value tells you whether observed data are unusual under a null model. It does not tell you whether your benchmark was large enough to detect a small-but-real improvement. So p = 0.71 can mean either: there is truly no effect, or there is a small effect, but your benchmark has low detection power.…