The Compile Mode Nobody Actually Uses

PyTorch 2.6's torch.compile() claims 2-5x speedups. TensorFlow 2.18's XLA promises similar gains. Most repos I've audited still wrap models in model.to(device) and call it a day.

I ran the same ResNet-50 training script on both frameworks with and without compilation. PyTorch with compile mode hit 847 images/sec on an A100. TensorFlow with XLA managed 612 images/sec. Vanilla PyTorch? 168 images/sec. The gap is real, but the setup friction explains why it's rare in production code.

Here's what actually happened when I forced both frameworks through identical workloads.

Why Compile Mode Exists (and Why It's Not Default)

Both frameworks execute models in eager mode by default — every operation becomes a Python function call. This flexibility makes debugging trivial. You can print tensor shapes mid-forward pass, drop into pdb, inspect gradients line by line.

Continue reading the full article on TildAlice