Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
1 / 5
0

Reranker Fine-Tuning on Click Data: When Off-the-Shelf Stops Winning

DEV Community·Gabriel Anhaia·25 days ago
#za9duFUs
#rag#ai#machinelearning#search#click#reranker
Reading 0:00
15s threshold

Book: RAG Pocket Guide: Retrieval, Chunking, and Reranking Patterns for Production Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub You wired up a RAG pipeline, picked bge-reranker-v2-m3 off Hugging Face, plugged it in after the bi-encoder, and watched hit rate jump eight points. Then you stopped touching it. Two quarters later the domain has drifted. The corpus is twice the size, half the queries are jargon the reranker has never seen, and the support team is filing tickets faster than the model is winning them. Recall@5 still looks fine. Users don't. This is the moment off-the-shelf rerankers stop being free wins. They were trained on MS MARCO and a handful of public IR datasets. Your queries are about your billing flow, your part numbers, your internal acronyms.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More