You built a RAG pipeline. You tested it manually with a few queries, got reasonable answers, and shipped it. Your stakeholders are happy. The chatbot returns answers that sound authoritative. But here is the uncomfortable question: do you actually know if it's right? Not "does it return something", but does it return the correct thing, consistently, across the full distribution of queries your users will actually ask? If you can't answer that with a number, you don't have a production system. You have a demo with a deployment URL. This article is for senior engineers who build RAG systems and want to know if they actually work. We will walk through a concrete RAG example - a pipeline over a corporate annual report - and build the testing layer that most teams skip entirely. The code is real and runnable. The failures are not hypothetical. All code in this article is available in the companion GitHub repository: github.com/nunombispo/rag-pipeline-article .…