Menu

Post image 1
Post image 2
1 / 2
0

I Benchmarked 4 Lightweight Transformers for Fault Detection. Here's What Survived.

DEV Community: machinelearning·Disha Patel·1 day ago
#4fMq2a0h
#dev#ai#python#benchmark#tinybert#failure
Reading 0:00
15s threshold

Everyone talks about deploying ML on edge devices. Very few people show what happens when you actually try. I ran a full benchmark of four lightweight transformer models - DistilBERT, MobileBERT, TinyBERT-6L, and TinyBERT-4L — against traditional ML baselines on three real-world fault detection datasets. The Setup NASA C-MAPSS : Turbofan engine degradation (20,631 samples, 15% failure rate) SECOM : Semiconductor manufacturing (1,567 samples, 6.6% failure rate) UCI Predictive Maintenance : Industrial machine failure (10,000 samples, 3.4% failure rate) All experiments ran on a T4 GPU with consistent hyperparameters. The Results Model F1 Size CPU Latency XGBoost 87.9% 0.5 MB 0.002 ms TinyBERT-4L 87.8% 55 MB 18 ms DistilBERT 87.6% 255 MB 138 ms MobileBERT: The Surprise Failure MobileBERT — specifically designed for mobile deployment — scored 0% F1 on every dataset . It predicted the majority class for every sample across all configurations.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More