FinBERT Wins by 31 Points. But Here's When It Doesn't Matter.

I ran both models on 500 earnings call transcripts, extracting sentiment around forward guidance statements. FinBERT (ProsusAI/finbert) hit 76% agreement with analyst consensus ratings. DistilRoBERTa-base topped out at 45%. That's a 31 percentage point gap.

But before you dismiss DistilRoBERTa entirely — inference latency tells a different story. DistilRoBERTa processes a typical earnings call (8000 tokens) in 1.2 seconds on CPU. FinBERT takes 4.7 seconds for the same input. If you're building a real-time trading signal that needs to react within seconds of transcript release, that 3.5 second difference compounds fast across multiple concurrent calls.

The question isn't "which model is better" — it's "how much accuracy do you actually need, and what's your inference budget?"

The Test Setup: 500 Earnings Call Transcripts

Continue reading the full article on TildAlice