Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation combines information retrieval with generation so a model can answer using selected external evidence at request time.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/retrieval-augmented-generation/
01What it means
Retrieval-augmented generation (RAG) is an architecture in which a system retrieves relevant material from an external collection and supplies it to a generative model as context for an output. Retrieval provides non-parametric, updateable evidence; the model synthesizes a response. RAG is distinct from fine-tuning and does not itself guarantee grounded or correct answers.[1][2]
- 01Governed sourcesMetadata and provenance
- 02IndexSemantic, lexical, hybrid
- 03Retrieve + rerankPermitted evidence
- 04Context + modelBounded prompt
- 05AnswerCitations or abstention
Retrieved context does not guarantee a grounded answer.
02Why it matters
RAG can connect a general model to current or proprietary knowledge without retraining it, support citations, and make source updates operationally simpler. It also splits quality across ingestion, retrieval, context construction, generation, and presentation, giving teams more layers to evaluate and debug.
03When to use it
Use RAG when responses depend on a sizeable, changing, or attributable corpus and users benefit from synthesis. Prefer ordinary search when source discovery is the goal, a database query for exact structured facts, or fine-tuning when the main need is stable behavior rather than new knowledge.
04How it works
- Ingest governed sources, divide them into meaningful units, attach metadata, and preserve provenance.
- Represent and index content for semantic, lexical, or hybrid retrieval under access controls.
- Transform a request into a query and retrieve, filter, or rerank candidate evidence.
- Construct bounded context and ask the model for an answer with citations or abstention when evidence is insufficient.
- Evaluate retrieval and generation separately and end to end, then monitor freshness, latency, cost, and failures.
05A practical example
A product-operations assistant answers process questions from approved handbook pages. It retrieves passages with department permissions, reranks by authority and date, requires source links, and returns no supported answer when relevant evidence is absent instead of inventing a policy.
06Where it breaks
RAG can retrieve irrelevant, stale, conflicting, or malicious content; the model may ignore evidence or cite it incorrectly. Chunking loses context, broad retrieval can leak data, and added stages increase latency and operational complexity. It does not update model parameters or eliminate hallucination.
Common mistakes
- Measuring answer style while failing to measure whether the right evidence was retrieved.
- Calling every prompt with attached text RAG without a governed retrieval process.
- Using vector similarity as the only signal for authority, recency, and permissions.
- Assuming RAG and fine-tuning are interchangeable ways to add factual knowledge.
07Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksAdvances in Neural Information Processing Systems · Peer-reviewed researchRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Retrieval-Augmented Generation for Large Language Models: A SurveyarXiv · Research surveyRetrieval-Augmented Generation for Large Language Models: A Survey