Menu

Post image 1
Post image 2
Post image 3
1 / 3
0

When Your Embeddings Stop Distinguishing Anything

DEV Community·Gabriel Anhaia·about 1 month ago
#WmQ0PxMX
#rag#ai#pairs#embedding#similarity#random
Reading 0:00
15s threshold

Book: RAG Pocket Guide Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub The provider's status page is green. Your error rate is flat. Latency is fine. And yet your RAG retrieval has gone soft. Top-1 hit rate roughly halved overnight, and nobody touched the index. You start digging and notice something strange. You pick a query and a known-good answer chunk and compute cosine similarity. It comes back at 0.987. Good, right? Then you pick a query and an obviously unrelated chunk. That one is 0.984. Then you pick two random chunks from your corpus. Also 0.98. Every pair you sample is sitting in the same narrow band near 1.0. The embedding model is still answering. It's still returning 1536-dim vectors. It's still passing every health check you have. But it has stopped distinguishing anything.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More