When the Catalog Points to the Wrong Shelf

Opening Scene

A patron who gets sent to entirely the wrong shelf doesn’t get an entirely wrong answer by accident — there’s usually a specific, identifiable reason the catalog search failed: the patron’s search terms didn’t match how the material was actually cataloged, the truly relevant book was miscataloged in the first place, or the collection simply doesn’t contain what the patron needs at all. Each of these is a genuinely different problem, requiring a genuinely different fix. RAG retrieval failures follow this same diagnosable pattern.

In Plain English

Retrieval failure modes fall into several distinct, well-documented categories: the query and relevant content are phrased too differently for embeddings to connect them (a vocabulary mismatch), the relevant content was chunked poorly and split across boundaries (connecting to Article 4), the knowledge source genuinely doesn’t contain the needed information at all, or the relevant content exists but ranks too low to be included in the top-k results, covered in Article 8. Correctly diagnosing which failure mode occurred is essential to applying the right fix.

The Old Way

Before these failure modes were well documented, diagnosing a poor RAG result was often a more unstructured, trial-and-error process:

A poor RAG answer was often attributed vaguely to “the retrieval didn’t work,” without pinpointing the actual specific cause.
Fixes were sometimes applied without confidence that the actual root cause had been correctly identified, risking wasted effort or a fix that didn’t address the real problem.
The specific, recurring categories of retrieval failure weren’t yet a well-established, shared diagnostic framework across the field.

Accumulated practical RAG experience has produced a genuine, shared taxonomy of common failure modes, making diagnosis considerably more systematic.

What’s Changing (and Why AI Is the Reason)

This series’ earlier articles — chunking in Article 4, top-k tuning in Article 8, relevance ranking in Article 9 — together form a genuine diagnostic checklist for identifying exactly why a retrieval failed.
Query rewriting and expansion techniques have emerged specifically to address vocabulary mismatch, reformulating a user’s query to better match how relevant content is actually phrased.
Systematic RAG evaluation, covered in Article 18, has made it considerably easier to detect and categorize retrieval failures at scale, rather than relying on manually noticing individual bad results.

The Metaphor, Fully Extended

The Library	Retrieval Failure Concept
A patron’s search terms not matching how material was actually cataloged	A vocabulary mismatch between the query and relevant content’s phrasing
A relevant book miscataloged or split awkwardly across sections	Relevant content chunked poorly and split across boundaries
The collection simply not containing what the patron needs	The knowledge source genuinely lacking the needed information
A relevant book that exists but never makes it onto the recommended list	Relevant content that exists but ranks too low for the top-k cutoff

For Beginners: What to Actually Do

Practice diagnosing a poor RAG result by working through this article’s failure-mode categories systematically, rather than guessing at a fix.
Learn to distinguish “the content doesn’t exist in the knowledge source” from “the content exists but wasn’t retrieved,” since these require genuinely different responses.
Get comfortable using query rewriting as a first response to a suspected vocabulary mismatch.

For Practitioners and Leaders: The Deeper Layer

Build a shared, documented failure taxonomy for your organization’s RAG systems, connecting directly to this article’s categories.
Invest in query rewriting and expansion techniques specifically to address vocabulary mismatch, one of the most common, addressable failure modes.
Use systematic evaluation, covered in Article 18, to detect and categorize retrieval failures at scale, rather than relying on anecdotal spot-checks.

Quick Recap

Retrieval failures fall into distinct categories: vocabulary mismatch, poor chunking, missing content, and low-ranking relevant content.
Correctly diagnosing which failure mode occurred is essential to applying the right, targeted fix.
This series’ earlier articles together form a genuine diagnostic checklist for identifying retrieval failures.
Query rewriting specifically addresses vocabulary mismatch, one of the most common failure categories.

Where This Fits in the Series

Article 10 covered diagnosing why retrieval pointed to the wrong shelf. Article 11 covers a second layer of judgment specifically designed to catch these mistakes before they reach the model.