Menu

Post image 1
Post image 2
Post image 3
1 / 3
0

When the Reranker Hurts: Recall@5 Cases Where Two-Stage Retrieval Loses to One

DEV Community·Gabriel Anhaia·about 1 month ago
#HfIB3aD1
#ai#rag#llm#reranker#query#encoder
Reading 0:00
15s threshold

Book: RAG Pocket Guide Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub A team I spoke with was debugging a sudden recall regression. Their RAG pipeline had been fine for months. Then product moved the search box from the docs site to an internal admin tool, and answers got worse overnight. Same corpus and embedding model, and the same bge-reranker-v2-m3 in stage two. Recall@5 on their golden set dropped from 0.83 to 0.71. The instinct was that something in the index drifted. The actual cause was simpler. The new query distribution was short, typo-heavy admin queries: "refnd 14d policy" , "webhook timeout pcfg" . The cross-encoder reordered the candidate set worse than the bi-encoder alone. Pulling the reranker out raised recall@5 back to 0.82.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More