A human-review path should identify the exact unresolved field, evidence IDs, proposed next check, and any action that was withheld. Route cases by consequence and reversibility: low-impact formatting can proceed with a warning, while a payment or account change may require a human. The reviewer needs a compact packet, not the entire conversation transcript. The application should prevent a stale model recommendation from being executed after the underlying data changes.
Make this comfortable
Human handoff: preserve evidence and the reason for uncertainty
Decision in practice
A credit assistant finds two invoices with the same order number and different account owners. It sends a review packet containing both invoice IDs, the conflict, policy clause, and the proposed account lookup. It does not issue the credit. The reviewer resolves the owner and returns a signed decision; the application checks that the invoices have not changed before any effect. If the case is edited during review, the packet expires and must be rebuilt. The assistant's earlier recommendation is never treated as a standing approval.
Review packet: case CR-419, conflict invoice-71 vs invoice-88.
Withheld effect: issue_credit.
Next check: verify account ownership in the ledger.
Expiry: invalidate if invoice version changes before approval.Performance and operating cost
Escalation adds human queue time, so route only cases where uncertainty or consequence justifies it. Measure referral rate, reviewer agreement, stale-packet rejection, and time to resolution. A low referral rate is not proof of success if the model hides uncertainty. Conversely, sending every case to review erases the workflow's value. A bounded evidence packet reduces reviewer reading time and limits private data exposure while preserving the facts needed to decide.
Common Mistakes
- Do not send a vague 'needs review' with no blocking fact.
- Do not execute an old recommendation after source data changes.
- Do not treat reviewer silence as approval.
Connected lessons
- Prompt Engineering
- Production prompt engineering
- Output contracts: parse a result and preserve an explicit unknown state
- Tool calls: validate intent and arguments before an external effect
- Metamorphic tests: verify behavior when harmless details change
- Prompt traces: connect an answer to its inputs, checks, and effects
- Project: regression-test a customer triage prompt
- Advanced prompt engineering decisions
Related implementation
Continue with: High-impact advice: route uncertain decisions to qualified review.
Continue with: Assessment prompts: separate feedback from final grading.
prompt engineering
tutorial