Find the questions your RAG pipeline will fail on, before your users do
ragProbe is a tool designed to identify questions that a RAG (Retrieval-Augmented Generation) pipeline will answer incorrectly, allowing developers to detect failures before users encounter them.
Background
RAG (Retrieval-Augmented Generation) is a technique that lets large language models answer questions by first pulling in relevant documents from a custom knowledge base — crucial for enterprise AI use cases where the model needs to reference private data. However, RAG pipelines often silently fail: the retriever component might miss the right document, or the generator might ignore retrieved context. ragProbe is an open-source tool that systematically tests a RAG pipeline and pinpoints which questions it gets wrong, so developers can fix weaknesses before users encounter them. This matters because most production RAG systems lack robust evaluation tooling, making it hard to know when a system is truly reliable.