# Foundation Model

A foundation model is a broadly trained model that can be adapted or prompted for many downstream tasks rather than built for only one narrow purpose.

- Type: term
- Domain: Engineering & architecture
- Level: Intermediate
- Last reviewed: 2026-09-26
- Original: [Soroush’s Product Wiki](<https://wiki.soroush.uk/topic/foundation-model/>)

## What it means

A foundation model is trained on broad data, usually at scale, and designed to support a range of downstream uses through prompting, retrieval, fine-tuning, or other adaptation. The term describes a reusable base and its role in an ecosystem, not a guarantee of quality. Large language models and multimodal models can be foundation models, while a task-specific model may not be one.

## Why it matters

A shared base can shorten development and enable capabilities that would be impractical to train from scratch. It also concentrates dependencies: limitations, data choices, licensing, security, cost, and changes in the base model can propagate into many products and user groups.

## When to use it

Consider a foundation model when broad learned capability transfers to the task and adaptation is cheaper or better than a narrow model or rules. Compare providers and smaller alternatives when privacy, control, latency, unit economics, specialized accuracy, or deployment constraints dominate.

## How it works

1. Define the downstream task, baseline, risks, deployment constraints, and evidence required.
2. Shortlist base models using capability, modality, context, governance, licensing, security, and cost criteria.
3. Prototype the least invasive adaptation—prompting first, then retrieval or fine-tuning when evidence supports it.
4. Evaluate end-to-end behavior on representative, edge, and adversarial cases rather than relying on general benchmarks.
5. Version the model and surrounding system, monitor provider changes, and retain migration or rollback options.

## A practical example

A document product compares two hosted foundation models and a smaller task model for contract clause extraction. The general model wins on varied wording, but the team uses structured output, a review queue, pinned versions, and a fallback because model updates affect downstream behavior.

## Where it breaks

Broad training does not ensure domain accuracy or safe use. Training data and model behavior may be opaque; adaptation can preserve underlying biases and vulnerabilities. Large models can create vendor dependence, substantial inference cost, and correlated failures across features.

## Common mistakes

- Using foundation model as a synonym for every large or generative model.
- Selecting from benchmark rank alone without testing the actual product workflow.
- Assuming retrieval or fine-tuning removes base-model risks.
- Building tightly around one provider without versioning or an exit path.

## Visualization

[View the visualization on the website](<https://wiki.soroush.uk/topic/foundation-model/#visualization-foundation-model>)

## Sources

1. [On the Opportunities and Risks of Foundation Models](<https://arxiv.org/abs/2108.07258>) — Stanford Center for Research on Foundation Models · Research report
2. [Machine Learning Glossary: Generative AI](<https://developers.google.com/machine-learning/glossary/generative>) — Google for Developers · Official technical glossary

## Related topics

- [Generative AI](<https://wiki.soroush.uk/topic/generative-ai/>)
- [Large Language Model \(LLM\)](<https://wiki.soroush.uk/topic/large-language-model/>)
- [Fine-Tuning](<https://wiki.soroush.uk/topic/fine-tuning/>)
