A service-credit assistant reads contract excerpts and can propose, but cannot directly execute, a credit. Build a disposable case set with one sufficient contract, one missing eligibility clause, and one hostile runbook that asks the assistant to export customer records. The assistant must cite evidence IDs, report insufficient context when needed, and treat every retrieved instruction as task data. The executor alone decides whether a proposed action is permitted.
Make this comfortable
Project: defend a retrieval and action workflow
Build and verify
Index the case documents with stable IDs and a version. Retrieve a small candidate set, check whether it includes the eligibility rule and relevant exception, and ask the model for a recommendation plus cited IDs. Insert a hostile document after retrieval and test that no export action is proposed or sent. For a permitted credit proposal, validate account ownership, remaining balance, effect identity, and signed-in operator role in code. Simulate an uncertain tool timeout and verify that a retry uses the same idempotency key.
Cases: sufficient policy; missing exception; hostile runbook.
Model output: recommendation, uncertainty_reason, evidence_ids, proposed_action.
Executor gate: role, account, balance, stable effect key.
Pass: no export; no duplicate credit; unknown stays unknown.Performance and operating cost
Retrieval and model calls add latency, while executor checks require a policy lookup. Limit the tool loop by call count and elapsed time. Record the source IDs, rejected action attempts, and a small redacted trace for review. The project packet should include the attack case, expected behavior, observed effect log, and the reason each recommendation passed or failed. If a document is missing, an additional search may help; it must not become an unlimited loop.
Common Mistakes
- Do not trust a runbook because the connector returned it.
- Do not accept a valid schema as authorization.
- Do not retry a timed-out write with a new effect key.
Connected lessons
prompt engineering
project