Menu

Post image 1
Post image 2
1 / 2
0

Benchmark: PyTorch 2.5 vs. TensorFlow 2.18 for Training Vision Transformers on 8x H100 GPUs

DEV Community·ANKUSH CHOUDHARY JOHAL·about 1 month ago
#JuovsYWN
#benchmark#code#tip#use#pytorch#tensorflow
Reading 0:00
15s threshold

\n Training a Vision Transformer (ViT-B/16) on ImageNet-21K used to take 3 days on 8x A100 GPUs. On 8x NVIDIA H100 SXM5 GPUs, we cut that to 14 hours with PyTorch 2.5—but TensorFlow 2.18? It stumbled out of the gate. Here’s the unvarnished benchmark data. \n\n 📡 Hacker News Top Stories Right Now New Integrated by Design FreeBSD Book (29 points) Microsoft and OpenAI end their exclusive and revenue-sharing deal (724 points) Talkie: a 13B vintage language model from 1930 (37 points) Three men are facing charges in Toronto SMS Blaster arrests (72 points) Is my blue your blue? (289 points) \n\n \n Key Insights \n \n* PyTorch 2.5 achieves 1,420 images/sec throughput on 8x H100 for ViT-B/16, 18% faster than TensorFlow 2.18’s 1,190 images/sec \n* TensorFlow 2.18’s XLA compilation adds 22 minutes of warmup per training run, vs.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More