Project: select and verify a reasoning pattern

Choose the cheapest useful method

For shipment SH-731, a deterministic calculation is already possible; compare it against five-candidate self-consistency only to measure whether sampling adds value. For the dispatch schedule, use a bounded branch search with a three-decision depth and a two-plan beam, and reject any plan that exceeds a van limit or cold window. For pump PM-84, use an observe-and-act read loop that stops when supervisor sign-off is absent. Add a fourth case, order OR-427, whose credit decision depends on contract version, signed arrival, and an exclusion rule. Resolve those dependencies in order. A fifth case asks for a cold-storage invoice total; extract typed fields, validate the charge base, then pass the expression to a restricted evaluator.

Check the decisions

Record source revisions, one expected outcome per case, and a benign control where a direct response is enough. Count correct final decisions, unsupported claims, missed stop conditions, total model calls, tool calls, and review time. A model vote cannot substitute for the current manifest; a running program cannot establish that it used the right charge base; and a plausible branch cannot release a pump. Compare the candidate workflow with the direct baseline on the same evidence and hold out at least one case per failure class from prompt editing.

SH-731: direct arithmetic vs five-candidate vote; verify manifest revision.
Dispatch: depth=3, beam=2; reject capacity and cold-window failures.
PM-84: at most 4 reads; missing sign-off means stop.
OR-427: contract -> arrival -> exclusion -> credit.
Invoice: typed fields -> restricted decimal evaluator.
Release: correct checked result, no unauthorized effect, recorded cost.

Performance and operating cost

For C cases and V variants, offline comparison needs O(CV) model evaluations. A bounded search may expand O(DWB) partial plans for depth D, beam W, and branch limit B. Sequential subproblems and tool observations add latency. Record each call because a more accurate method may still be unsuitable under a strict response-time or cost limit. The evaluation packet should preserve enough redacted evidence to reproduce decisions without storing private payloads. A failed critical permission or safety check blocks release even if the average score improves.

Common Mistakes

• Do not select the most elaborate pattern before measuring a direct baseline.
• Do not let a model score replace a hard feasibility or authorization check.
• Do not count a computed number as verified when its source fields are wrong.

Connected lessons

• Self-consistency: sample answers, then verify the winner
• Tree of Thoughts: branch only where a decision can be checked
• ReAct: alternate tool actions with checked observations
• Least-to-most prompting: solve smaller dependencies first
• Program-aided prompting: make computation executable and bounded
• Evaluation sets: measure the failure cases that matter
• Negative controls: test the answer that should not be produced
• Reasoning patterns and operating limits