A structured-output retry starts only after a parser identifies a specific format defect. Return the schema and the parser error to the model, keep the original task and evidence fixed, and cap attempts. Revalidate the entire response after each repair. If the first response chose review because evidence was missing, a retry must not convert that into approve merely to satisfy a required field. The safest design separates parse repair from semantic re-evaluation; when the business fields change, treat the result as a new candidate and rerun the decision gates.
Make this comfortable
Structured outputs: repair format without changing the decision
Decision in practice
A reimbursement assistant returns a JSON object for claim CL-509 with decision review, but the closing brace is missing. The application requests one repair with the same evidence packet and states that the decision and evidence IDs must remain unchanged. The repaired object parses, yet it changes review to approve. The structural error is fixed, but the semantic change fails the gate and the case goes to a human reviewer. A second case has a duplicate key; the parser must reject it under the application's chosen JSON policy rather than accepting an arbitrary last value.
Original decision: review; evidence_ids: [RC-52].
Parser error: closing brace missing.
Repair contract: return valid JSON only; preserve decision and evidence_ids.
Validation: parse -> schema -> unchanged business fields -> evidence gate.
Budget: one repair; otherwise review with original error recorded.Performance and operating cost
A retry adds another model call and can roughly double generation cost for affected requests. If p is the parse-failure share, the expected extra call count under one retry is p per request; measure this rather than assuming retries are rare. Parsing and field comparison are O(L) in the response length. A retry loop without a cap can hide a broken prompt behind cost and latency. Log both the original defect and any semantic drift across repairs.
Common Mistakes
- Do not accept a repaired object solely because it parses.
- Do not allow repair to erase an explicit review state.
- Do not retry forever when the schema and prompt disagree.
Connected lessons
- Prompt patterns
- Prompt Engineering
- Output contracts: parse a result and preserve an explicit unknown state
- Streaming responses: validate the complete artifact before an effect
- Tool catalogs: describe eligibility, inputs, and effects
- Tool plans: separate independent reads from dependent actions
- Conversation checkpoints: resume from verified state
- Tool results: keep returned text in the data lane
- Tool effects: reconcile receipts before retrying
- Project: recover a tool workflow without duplicate effects
- Agent workflow decisions
Related implementation
prompt engineering
agent workflows