Contrastive examples are paired cases that differ in one material fact and require different outcomes. They teach the model which condition matters without burying the rule in many easy positives. Include an explicit uncertain case when evidence is missing. Keep examples representative and free of final holdout cases. Order can influence outputs, so test reordered examples and paraphrased inputs. A few-shot prompt is a task aid, not a permission system or a replacement for validating the final answer.
Make this comfortable
Few-shot examples: teach the boundary with near misses
Decision in practice
A renewal prompt contains two approved examples with recent inspections. It then approves RN-284 despite an inspection older than the 47-day rule. The author adds a matched pair: a vehicle inspected 46 days ago with no hold is eligible; another inspected 48 days ago is not. A third example with a missing hold record returns review. The team freezes these demonstrations and tests a separate set with ages near the boundary, changed wording, and the same facts in a different order. Passing the prompt examples themselves does not count as a release result.
Example A: inspection age 46 days, hold clear -> eligible.
Example B: inspection age 48 days, hold clear -> ineligible.
Example C: inspection age 21 days, hold unknown -> review.
Held-out checks: ages 47 and 49; reordered descriptions; new asset IDs.
Final decision still validated by rule and current records.Performance and operating cost
Adding examples increases prompt tokens on every call. Three boundary examples can be cheaper and more informative than a long list of routine positives, but that must be measured on a held-out set. If E examples each average T tokens, the prompt cost adds roughly O(ET) tokens per request. Selection and labeling of near misses take editorial effort. Watch for example order effects and duplicated cases across prompt and test data. The improvement target is behavior on unseen boundary cases, not memorization of the demonstrations.
Common Mistakes
- Do not fill every demonstration with the same easy label.
- Do not use a release holdout case as a prompt example.
- Do not assume an example grants permission for an external action.
Connected lessons
- Prompt patterns
- Prompt Engineering
- Few-shot prompting: select examples that cover decisions, not just easy cases
- Evaluation leakage: keep the release test independent
- Reasoning summaries: show checkable grounds, not invented certainty
- Multiple candidates: filter invalid answers before ranking
- Critique loops: require a named defect and a stopping rule
- Context distillation: shorten input without losing governing exceptions
- Negative controls: test the answer that should not be produced
- Project: verify a contract-renewal prompt at the boundary
- Reasoning and evidence checks
Continue with: Classification prompts: write the label boundary first.
prompt engineering
evaluation