Context distillation replaces a large retrieved bundle with a smaller evidence packet for the final prompt. The distillation stage must preserve the source version, decisive clause, limiting exception, and uncertainty; otherwise it can produce a shorter but less trustworthy answer. Treat the compact packet as a derived artifact with a link back to the full record. Test whether a reviewer can reconstruct each generated claim from the original source. Use deterministic metadata filters before semantic compression when possible, and refuse to fill missing clauses from memory.
Make this comfortable
Context distillation: shorten input without losing governing exceptions
Decision in practice
A renewal policy spans 31 pages. The relevant clause allows renewal after inspection, but an appendix suspends that rule for units with unresolved safety holds. A summary that keeps the allowance and drops the appendix would approve RN-284 incorrectly. The distiller therefore carries both clause IDs, the policy version, and a note that hold status must be checked. A reviewer opens the original appendix for a disputed case. The test packet includes a policy update that moves the exception to a different section; the system must refresh the compact packet instead of reusing a stale one.
Compact packet for RN-284:
POL-64 v4 clause 8: inspection must be recent.
POL-64 v4 appendix C: unresolved safety hold blocks renewal.
Unknown: hold status for RN-284.
Source map: each item retains record ID, section ID, version, and retrieval time.
Decision: review until hold status is verified.Performance and operating cost
Reading a large bundle costs tokens and latency; distillation adds a stage but may reduce later repeated context. If N source chunks are scanned, initial selection is at least O(N) in the number of chunks, and a source map takes O(K) space for K retained claims. Measure exception loss and stale-packet errors, not just token reduction. Cache a packet only while its source versions remain valid. A compressed note that cannot identify its governing source is too weak for a consequential decision.
Common Mistakes
- Do not keep the main rule while dropping its exception.
- Do not discard source IDs during compression.
- Do not reuse a packet after its policy version changes.
Connected lessons
- Prompt patterns
- Prompt Engineering
- Document chunks: preserve the clause and its governing exception
- Long context: make inclusion and truncation testable
- Reasoning summaries: show checkable grounds, not invented certainty
- Multiple candidates: filter invalid answers before ranking
- Critique loops: require a named defect and a stopping rule
- Few-shot examples: teach the boundary with near misses
- Negative controls: test the answer that should not be produced
- Project: verify a contract-renewal prompt at the boundary
- Reasoning and evidence checks
Continue with: Memory prompts: bind namespace, identity, and provenance.
prompt engineering
evaluation