Retrieval: giving the model evidence
A retrieval system finds relevant material and places it where a language model can use it.
In this lesson
Retrieval changes the available context; it does not automatically rewrite the model’s weights or guarantee a true answer.
Two places information can come from
A model’s parameters reflect its training. An application can also supply external information at inference time: a manual, a document excerpt, or a search result. Retrieval-augmented generation connects a retrieval step to generation. Lewis and colleagues’ 2020 RAG paper describes a learned architecture combining parametric and external memory. The term is now also used more broadly for retrieve-then-generate pipelines.
That broader pattern does not require the exact architecture in the original paper. A practical system might use keyword search, vector similarity, or both to find candidate passages. It then passes selected material into a model’s context. The model generates a response conditioned on what was actually supplied.
Follow the handoff
A typical pipeline receives a question, forms a search query, retrieves candidate documents, selects useful passages, and constructs the model input. The final answer may include citations pointing to those passages. Each handoff can lose information: a poor query misses the right document; a bad excerpt drops the qualifying sentence.
The local demonstration uses a tiny, disclosed document collection and a deterministic lookup workflow. It illustrates the handoff, not a live search engine or a trained retrieval model. When the collection lacks an answer, the honest outcome is to show that missing evidence rather than fabricate a source.
The source enters through the context.
- QuestionWhat are the hours?
- RetrieverFind a passage
- Model inputQuestion + passage
- GenerationCompose a response
Evidence is useful only if it supports the claim
A citation is a pointer, not a truth stamp. The cited passage must actually support the answer, be appropriate for the question, and be current enough when time matters. An answer can cite a document while reversing its conclusion or extending beyond what it says. Evaluation should inspect the relationship between claim and passage.
External text can also contain misleading instructions. A retrieved page that says “ignore the user” is still source material, not an authorized instruction to the application. Separating source content from control instructions is a systems problem. Relevant retrieval and safe tool orchestration solve different parts of it.
What retrieval changes and what it leaves alone
Updating a document index can make new information available without retraining the language model. That is one reason retrieval is attractive for changing manuals or private collections. Access controls still matter: a search result should contain only material the user is allowed to receive.
Retrieval does not guarantee that the model reads every passage correctly, resolves contradictions, or knows when the index is incomplete. For a useful test set, include answerable questions, questions with conflicting evidence, and questions that cannot be answered from the collection. A system that always produces an answer can look fluent while failing the last category completely.
A dated document beats a guess
In an invented library, a policy document says a workshop begins at 10:00 on 3 May. Retrieval can supply that passage and its date. If a newer policy changes the time, the system needs the newer evidence; the model’s confident phrasing cannot resolve the conflict by itself.
Give the model a source
A deterministic local workflow with fictional documents; no live model or web search.
Every LLM automatically searches the web before answering.
Retrieval requires an external system and an actual retrieval operation. Many model calls have neither.
Take it with you
- Retrieval supplies context from an external collection.
- Inspect whether evidence supports each claim.
- Missing evidence should be an explicit outcome.
Check your understanding.
Choose an answer, then see the explanation. You can try again.
Practice only. Your answer is not stored or sent anywhere.
Tool use: letting a system act
Continue when you’re ready. Marking this lesson as read is optional.
Sources & further reading
Primary references for this lesson. The explanation and examples above are original.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks ↗Patrick Lewis et al. · 2020-05-22 · checked 13 Nov 2026