What the Librarian Actually Does

Opening Scene

A librarian helping a patron does two genuinely distinct jobs, one after the other. First, understanding the question well enough to locate the actually relevant books on the actually relevant shelves — a search and retrieval task. Second, once those specific books are in hand, actually reading and synthesizing them into a clear, direct answer to the patron’s original question — a genuinely different, generative task. Conflating these two steps, or doing either one poorly, breaks the whole system.

In Plain English

RAG consists of two genuinely distinct phases: retrieval, which finds the most relevant pieces of content from a knowledge source given a query, and generation, which uses a language model to synthesize a coherent answer from that retrieved content. Retrieval quality and generation quality are separate concerns, each with its own failure modes — a system can retrieve perfectly relevant documents and still generate a poor answer, or retrieve poor documents and have the generation step produce a confident, well-written, but ultimately ungrounded response.

The Old Way

Before this two-phase structure was well understood, early attempts at grounding model output blurred these steps together:

Early approaches sometimes provided a model with an entire large document and asked it to find relevant information itself, without a genuinely separate, dedicated retrieval step at all.
Search and generation were often treated as a single, undifferentiated problem, rather than as two distinct phases each deserving its own dedicated attention and evaluation.
Debugging a poor RAG result was harder without a clear separation between “the retrieval found the wrong content” and “the generation used good content poorly.”

Recognizing retrieval and generation as genuinely separate phases, each independently measurable and improvable, represents real, hard-won practical wisdom.

What’s Changing (and Why AI Is the Reason)

Treating retrieval and generation as separate, independently evaluable phases has become standard RAG system design practice, letting practitioners diagnose exactly where a system is underperforming.
This series follows this two-phase structure directly: Articles 4 through 11 cover retrieval mechanics in depth, while Articles 12 onward cover generation, grounding, and the broader system considerations that follow.
As RAG systems have matured, specialized tooling has emerged for each phase independently — vector databases and rerankers for retrieval, prompt engineering techniques for generation — reflecting genuine, separate engineering disciplines.

The Metaphor, Fully Extended

The Library	RAG’s Two-Phase Concept
A librarian locating the actually relevant books on the shelves	The retrieval phase, finding the most relevant content
A librarian reading and synthesizing those books into a clear answer	The generation phase, synthesizing a coherent answer from retrieved content
Locating the right books but explaining them poorly	Good retrieval paired with poor generation
Explaining beautifully from the wrong books entirely	Confident generation grounded in poorly retrieved content

For Beginners: What to Actually Do

Practice mentally separating any RAG failure into “was the retrieved content actually relevant?” and “did the generation use that content well?”
Learn to evaluate retrieval quality independently from generation quality, since a good RAG system requires both to work well.
Get comfortable with the idea that improving one phase doesn’t automatically fix a problem originating in the other.

For Practitioners and Leaders: The Deeper Layer

Build separate evaluation metrics for retrieval quality and generation quality, connecting directly to the RAG evaluation practices covered in Article 18.
Invest in dedicated tooling for each phase, recognizing them as genuinely distinct engineering disciplines requiring different expertise.
Diagnose RAG system failures by first isolating which phase is actually underperforming, rather than treating the whole system as one opaque block.

Quick Recap

RAG consists of two genuinely distinct phases: retrieval (finding relevant content) and generation (synthesizing an answer from it).
Each phase has its own failure modes, and a system needs both to work well to produce a good result.
Treating these as separate, independently evaluable phases is standard, essential RAG system design practice.
Specialized tooling has emerged for each phase, reflecting genuinely distinct engineering disciplines.

Where This Fits in the Series

Article 2 covered RAG’s two genuinely distinct phases. Article 3 looks at what relying purely on a model’s memorized training looked like before retrieval entered the picture.