On Thursday I wrote about Karpathy's autoresearch , the 630-line training loop that runs 100 ML experiments overnight on a single GPU while you sleep. The post generated a lot of conversation, and most of it centered on the automation: agents doing research, models training themselves, the future of AI development as a lights-out factory. But there's a thing in autoresearch that deserves more attention than the automation, something that explains why this particular loop produced real results while so many other "autonomous AI" projects produce noise. And it has nothing to do with the agent, the GPU, or the training code. It's the metric. Why Autoresearch Actually Works Autoresearch uses a single evaluation criterion: validation bits per byte (val_bpb). Lower is better. The metric is independent of vocabulary size, which means the agent can change the tokenizer, the embedding dimensions, the entire model architecture, and the comparison remains valid. Every five-minute experiment produces a number.…