Generative AI
Generative AI uses learned patterns to produce new text, images, audio, video, code, or other content in response to input.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/generative-ai/
01What it means
Generative AI is a class of AI models designed to generate synthetic content resembling patterns in their training data. A user or system supplies a prompt or other input, and the model produces a probable output rather than retrieving a guaranteed stored answer. Large language models are one kind of generative model; not all AI is generative and not all generated content is correct or original.[1][2]
- User + application contextTask, instructions, permissions, evidence, and examples
- Generative modelPredicts content from learned parameter patterns
- Draft outputText, image, audio, video, code, or structured data
- Deterministic checksSchema, policy, permissions, references, and limits
- Human judgmentReview, edit, reject, or escalate based on stakes
- Product actionPresent, store, publish, execute, or withhold
Evaluation and observed failures refine context, controls, or model choice
Generated output is probabilistic and may be unsupported, biased, unsafe, or derivative; fluency is not evidence of truth or authorization.
02Why it matters
Generative systems make flexible natural-language and media interfaces practical, but variability and plausible errors alter how products must be specified, tested, explained, and controlled. They also introduce concerns about provenance, intellectual property, privacy, misuse, and the cost of reviewing output.
03When to use it
Use generative AI for drafting, transformation, exploration, summarization, or assistance when output can be checked and recovery is easy. Apply stronger evidence, approval, and access controls—or do not use it—for high-impact decisions, sensitive data, irreversible actions, and tasks requiring exact factual or legal guarantees.
04How it works
- Define the content task, user value, acceptable variation, and consequences of a wrong output.
- Select a model and grounding, prompting, or adaptation approach appropriate to the modality and domain.
- Constrain inputs, data access, output format, tools, and permissions according to the threat model.
- Evaluate representative and adversarial cases for usefulness, factuality, safety, bias, and consistency.
- Design disclosure, review, correction, fallback, feedback, and monitoring into the user journey.
05A practical example
A research repository offers generated summaries of approved interview notes. Each summary links back to source passages, carries an AI label, can be edited, and is evaluated for unsupported claims; raw restricted studies remain outside the model's retrieval scope.
06Where it breaks
Outputs are sampled from learned distributions and may be false, stereotyped, unsafe, or derivative. Model and vendor behavior can change. Grounding and guardrails reduce some risks but do not guarantee truth or compliance, and review effort can erase expected productivity gains.
Common mistakes
- Treating generated content as retrieved fact or original user evidence.
- Evaluating only impressive examples instead of a defined distribution of real tasks and failures.
- Uploading sensitive or copyrighted material without authority and data controls.
- Hiding AI involvement where users need it to judge, verify, or contest an output.
07Sources
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNational Institute of Standards and Technology · Government frameworkArtificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
- Machine Learning Glossary: Generative AIGoogle for Developers · Official technical glossaryMachine Learning Glossary: Generative AI