# Retrieval-Augmented Generation \(RAG\)

Retrieval-augmented generation combines information retrieval with generation so a model can answer using selected external evidence at request time.

- Type: framework
- Domain: Engineering & architecture
- Level: Intermediate
- Last reviewed: 2026-09-26
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/retrieval-augmented-generation/>)

## What it means

Retrieval-augmented generation \(RAG\) is an architecture in which a system retrieves relevant material from an external collection and supplies it to a generative model as context for an output. Retrieval provides non-parametric, updateable evidence; the model synthesizes a response. RAG is distinct from fine-tuning and does not itself guarantee grounded or correct answers.

## Why it matters

RAG can connect a general model to current or proprietary knowledge without retraining it, support citations, and make source updates operationally simpler. It also splits quality across ingestion, retrieval, context construction, generation, and presentation, giving teams more layers to evaluate and debug.

## When to use it

Use RAG when responses depend on a sizeable, changing, or attributable corpus and users benefit from synthesis. Prefer ordinary search when source discovery is the goal, a database query for exact structured facts, or fine-tuning when the main need is stable behavior rather than new knowledge.

## How it works

1. Ingest governed sources, divide them into meaningful units, attach metadata, and preserve provenance.
2. Represent and index content for semantic, lexical, or hybrid retrieval under access controls.
3. Transform a request into a query and retrieve, filter, or rerank candidate evidence.
4. Construct bounded context and ask the model for an answer with citations or abstention when evidence is insufficient.
5. Evaluate retrieval and generation separately and end to end, then monitor freshness, latency, cost, and failures.

## A practical example

A product-operations assistant answers process questions from approved handbook pages. It retrieves passages with department permissions, reranks by authority and date, requires source links, and returns no supported answer when relevant evidence is absent instead of inventing a policy.

## Where it breaks

RAG can retrieve irrelevant, stale, conflicting, or malicious content; the model may ignore evidence or cite it incorrectly. Chunking loses context, broad retrieval can leak data, and added stages increase latency and operational complexity. It does not update model parameters or eliminate hallucination.

## Common mistakes

- Measuring answer style while failing to measure whether the right evidence was retrieved.
- Calling every prompt with attached text RAG without a governed retrieval process.
- Using vector similarity as the only signal for authority, recency, and permissions.
- Assuming RAG and fine-tuning are interchangeable ways to add factual knowledge.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/retrieval-augmented-generation/#visualization-retrieval-augmented-generation>)

## Sources

1. [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](<https://arxiv.org/abs/2005.11401>) — Advances in Neural Information Processing Systems · Peer-reviewed research
2. [Retrieval-Augmented Generation for Large Language Models: A Survey](<https://arxiv.org/abs/2312.10997>) — arXiv · Research survey

## Related topics

- [Semantic Search](<https://wiki.soroush.uk/topic/semantic-search/>)
- [Grounding](<https://wiki.soroush.uk/topic/grounding/>)
- [Fine-Tuning](<https://wiki.soroush.uk/topic/fine-tuning/>)
