frameworkEngineering & architecture

Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation combines information retrieval with generation so a model can answer using selected external evidence at request time.

Download Markdown (.md)

01What it means

Retrieval-augmented generation (RAG) is an architecture in which a system retrieves relevant material from an external collection and supplies it to a generative model as context for an output. Retrieval provides non-parametric, updateable evidence; the model synthesizes a response. RAG is distinct from fine-tuning and does not itself guarantee grounded or correct answers.[1][2]

Retrieval + generation Retrieve evidence, then supply bounded context to the model
  1. 01Governed sourcesMetadata and provenance
  2. 02IndexSemantic, lexical, hybrid
  3. 03Retrieve + rerankPermitted evidence
  4. 04Context + modelBounded prompt
  5. 05AnswerCitations or abstention

Retrieved context does not guarantee a grounded answer.

02Why it matters

RAG can connect a general model to current or proprietary knowledge without retraining it, support citations, and make source updates operationally simpler. It also splits quality across ingestion, retrieval, context construction, generation, and presentation, giving teams more layers to evaluate and debug.

03When to use it

Use RAG when responses depend on a sizeable, changing, or attributable corpus and users benefit from synthesis. Prefer ordinary search when source discovery is the goal, a database query for exact structured facts, or fine-tuning when the main need is stable behavior rather than new knowledge.

04How it works

  1. Ingest governed sources, divide them into meaningful units, attach metadata, and preserve provenance.
  2. Represent and index content for semantic, lexical, or hybrid retrieval under access controls.
  3. Transform a request into a query and retrieve, filter, or rerank candidate evidence.
  4. Construct bounded context and ask the model for an answer with citations or abstention when evidence is insufficient.
  5. Evaluate retrieval and generation separately and end to end, then monitor freshness, latency, cost, and failures.

05A practical example

Example

A product-operations assistant answers process questions from approved handbook pages. It retrieves passages with department permissions, reranks by authority and date, requires source links, and returns no supported answer when relevant evidence is absent instead of inventing a policy.

06Where it breaks

RAG can retrieve irrelevant, stale, conflicting, or malicious content; the model may ignore evidence or cite it incorrectly. Chunking loses context, broad retrieval can leak data, and added stages increase latency and operational complexity. It does not update model parameters or eliminate hallucination.

Common mistakes

  • Measuring answer style while failing to measure whether the right evidence was retrieved.
  • Calling every prompt with attached text RAG without a governed retrieval process.
  • Using vector similarity as the only signal for authority, recency, and permissions.
  • Assuming RAG and fine-tuning are interchangeable ways to add factual knowledge.

07Sources

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksAdvances in Neural Information Processing Systems · Peer-reviewed researchRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  2. Retrieval-Augmented Generation for Large Language Models: A SurveyarXiv · Research surveyRetrieval-Augmented Generation for Large Language Models: A Survey
View saved topics

Start typing to search titles, definitions, categories, and frameworks.