Prompt Engineering
Prompt engineering is the systematic practice of designing, testing, and maintaining model inputs to improve behavior on a defined set of tasks.
Download Markdown (.md)Soroush’s Product Wiki · https://wiki.soroush.uk/topic/prompt-engineering/
01What it means
Prompt engineering applies iterative specification and evaluation to prompts, including instructions, context, examples, tool descriptions, and output requirements. It is an engineering practice when changes are versioned and measured against representative cases. It is not the search for a universal magic phrase, and it cannot compensate for the wrong model, missing evidence, unsafe architecture, or an unclear product goal.[1][2]
02Why it matters
Small prompt changes can affect usefulness, safety, cost, and consistency across many users. A disciplined practice makes these changes reviewable and reduces reliance on demos, intuition, and accidental behavior.
03When to use it
Use prompt engineering for model-based features after defining the task, evaluation criteria, and baseline. Move to retrieval, tools, fine-tuning, or deterministic code when the missing capability is evidence, action, learned behavior, or guaranteed logic rather than clearer instructions.
04How it works
- Define the task distribution, expected behavior, failure costs, and evaluation set before editing wording.
- Create a simple baseline prompt with clear instructions, relevant context, and a checkable output contract.
- Inspect failures and change one meaningful element, such as examples, structure, evidence, or tool description.
- Run regression and adversarial evaluations across important segments, models, and context lengths.
- Version prompts with model and configuration details, deploy gradually, and monitor real outcomes.
05A practical example
A finance assistant extracts invoice fields. The team begins with a schema and a labeled test set, analyzes date and currency failures, adds two representative examples, and accepts the revision only after it improves those cases without regressing multilingual invoices.
06Where it breaks
Prompt behavior is model- and version-dependent, and evaluation sets may miss real variation. Longer prompts increase cost and can add conflicts. Improvements can overfit test examples, while prompt-only controls remain vulnerable to untrusted input and model nondeterminism.
Common mistakes
- Optimizing a handful of favorite examples instead of a representative evaluation set.
- Changing prompt, model, and sampling settings together so the cause of improvement is unknown.
- Responding to every failure by making the prompt longer.
- Treating prompt instructions as authorization, validation, or a complete security control.
07Sources
- Prompt EngineeringOpenAI · Official documentationPrompt Engineering
- Introduction to PromptingGoogle Cloud · Official documentationIntroduction to Prompting