Repeated tool use can amplify a bad query, spend budget, or change state twice. Define maximum calls, elapsed time, data volume, and allowed tool sequence before execution. The loop should stop when the requested acceptance check is satisfied, when evidence is insufficient, or when a guardrail fails. Store a compact state record between calls: unresolved question, evidence IDs, completed effects, and remaining budget. The model should not be asked to remember whether it already wrote to an external system. Let the executor track that fact.
Make this comfortable
Tool loops: set budgets, state checks, and a stopping condition
Decision in practice
An asset-inventory assistant must locate the owner of server AS-914. It may query the inventory service twice and the on-call directory once. After the first query returns a stale owner and the second returns a current ownership record, it verifies the timestamp and stops. If the records conflict, it reports the conflict and routes to a human instead of making another 40 similar calls. A separate update-owner tool is outside this workflow. The trace records which calls ran and why the final answer used the current record.
Goal: current owner of AS-914.
Budget: 2 inventory reads, 1 directory read, 8 seconds.
Stop: current signed ownership record found; or conflict; or budget exhausted.
State: call IDs, evidence versions, unresolved reason, no write effects.Performance and operating cost
A call budget bounds direct tool expense at O(K) for K allowed calls, though returned data can still raise token cost. Too small a budget can create false abstentions; too large a budget can hide a retrieval defect behind repeated searches. Evaluate completion, abandonment, and cost per resolved case. Caching a verified result may reduce repeat reads, but it needs an expiry tied to the underlying record's freshness. Stop rules should be tested with missing, contradictory, and circular tool responses.
Common Mistakes
- Do not let the model decide its own unlimited tool budget.
- Do not use repeated search as a substitute for reporting a conflict.
- Do not keep effect history only in model context.
Connected lessons
- Prompt Engineering
- Prompt patterns
- Tool calls: validate intent and arguments before an external effect
- Prompt decomposition: split stages at verifiable handoffs
- Conversation memory: retain decisions without retaining every private detail
- Project: defend a retrieval and action workflow
- Prompt design decisions
Continue with: Tool effects: reconcile receipts before retrying.
Continue with: ReAct: alternate tool actions with checked observations.
prompt engineering
tutorial