Program-aided reasoning separates interpretation from computation. A model maps a natural-language request into a typed expression or small program; a deterministic evaluator calculates the result. This can reduce arithmetic slips, but it does not repair a wrong interpretation of the input. In production, prefer a limited expression schema or fixed calculation function over arbitrary generated code. Validate allowed operators, units, field names, ranges, and execution limits before evaluation. Never run free-form model output with unrestricted filesystem, network, or credential access.
Make this comfortable
Program-aided prompting: make computation executable and bounded
Operational case
A cold-storage invoice lists 37 pallet-days at 8.40 currency units each and a 6 percent handling charge on storage only. The assistant extracts quantities and constructs a typed calculation: storage is 310.80, handling is 18.648, and the final amount rounded once to two decimals is 329.45. The evaluator checks the units and the charge base. If the invoice instead says the handling charge applies to transport, the same arithmetic program would be wrong even though it runs perfectly. Source-field review remains a separate gate.
pallet_days=37; rate_per_pallet_day=8.40; handling_rate=0.06.
storage=37 * 8.40 = 310.80.
handling=310.80 * 0.06 = 18.648.
invoice_total=round(310.80 + 18.648, 2) = 329.45.
Allowed fields: pallet_days, rate, handling_rate; no arbitrary code.Performance and operating cost
A fixed arithmetic expression with T operators evaluates in O(T) time and O(T) space for a parsed expression tree; a dedicated function may use constant extra space for a fixed formula. The model call and extraction review usually dominate cost, while sandbox startup adds latency if arbitrary code is permitted. Prefer decimal arithmetic for money and define rounding once at the final boundary. Compare extracted inputs to the document before trusting the computed total; an exact evaluator will faithfully compute a mistaken field.
Common Mistakes
- Do not execute model-generated code with unrestricted privileges.
- Do not let a successful calculation certify the extracted business rule.
- Do not round every intermediate money value without a stated policy.
Connected lessons
- Prompt patterns
- Prompt Engineering
- Numeric prompts: let code calculate and the model explain
- Output contracts: parse a result and preserve an explicit unknown state
- Generated output: validate again at the destination boundary
- Self-consistency: sample answers, then verify the winner
- Tree of Thoughts: branch only where a decision can be checked
- ReAct: alternate tool actions with checked observations
- Least-to-most prompting: solve smaller dependencies first
- Project: select and verify a reasoning pattern
- Reasoning patterns and operating limits
prompt engineering
reasoning patterns