A retrieval-backed prompt receives excerpts selected by a search or ranking system. Relevance to the query is weaker than answer sufficiency: a passage can mention the right product while omitting the date, exception, or scope needed to answer. Give each excerpt a stable document ID, version, and section. Ask the model to identify the exact evidence required and to abstain when a required field is absent. Retrieval and generation should be measured separately, since rewriting the prompt cannot recover a passage the retriever never supplied.
Make this comfortable
Retrieved context: select sufficient evidence before writing the answer
Decision in practice
An operations assistant answers whether a 47-minute outage qualifies for a service credit. Search returns a current contract section with the threshold but no clause about planned maintenance, plus an older contract that mentions maintenance. The assistant must not combine those versions into a confident decision. It requests the current exception clause or returns unresolved with the missing clause named. The evaluation set includes both sufficient and insufficient retrieval bundles so the team can tell whether failures came from search coverage, version selection, or answer construction.
Question: Does incident INC-634 qualify for a service credit?
Required: current threshold, current exclusion clause, incident duration.
Retrieved: current threshold; old exclusion; duration 47 minutes.
Decision: unresolved; current exclusion clause missing.Performance and operating cost
Retrieval adds index storage, ranking time, and context tokens. Fetching more passages can improve recall yet increase latency and distraction; measure the rate of sufficient evidence at a fixed token budget. A model's own confidence is not a substitute for verifying the required fields. Keep superseded document versions distinguishable. When content is time-sensitive, a passage without an effective date should not silently outrank a current one merely because its wording matches the question.
Common Mistakes
- Do not treat a relevant excerpt as a complete answer.
- Do not merge clauses from different policy versions without stating the conflict.
- Do not blame prompt wording for a missing retrieved passage.
Connected lessons
- Prompt Engineering
- Prompt patterns
- Prompt context: separate instructions from retrieved material
- Output contracts: parse a result and preserve an explicit unknown state
- Evidence IDs: make generated claims auditable against supplied records
- Project: defend a retrieval and action workflow
- Prompt design decisions
Further prompt decisions
Test the task on changed inputs and record what the application actually accepted or rejected.
Next decision
Check whether the right facts reached the workflow and whether the result is safe at its destination.
- Retrieval queries: resolve references without adding facts
- Document chunks: preserve the clause and its governing exception
- Project: measure a retrieval-backed answer gate
Continue with: Retrieval prompts: enforce document permission first.
Continue with: Search prompts: define query intent and evaluation slices.
prompt engineering
tutorial