Chunking splits long documents for indexing, but a boundary can separate a general rule from the exception that limits it. Keep a stable chunk ID, document version, section heading, and parent-document link. Retrieval may find a short passage for ranking, then expand to neighboring or parent context before a consequential answer. The expansion must stay within the authorized document scope. Test boundary cases where the clause and exception sit on opposite sides of a chunk, and compare evidence sufficiency, not only search relevance.
Make this comfortable
Document chunks: preserve the clause and its governing exception
Decision in practice
A refund policy has a 44-day window in section 3 and an exception for opened equipment in section 4. Search returns section 3 for a request about an opened device. A prompt using only that chunk approves the refund incorrectly. The application expands to the parent policy segment, sees both sections, and sends the complete rule with IDs to the decision step. If section 4 is access-restricted, the workflow does not quote it from another tenant's index; it returns review because authorized evidence is incomplete.
Retrieved hit: policy-P17/section-3, 44-day window.
Parent expansion: section-4, opened-equipment exception.
Decision input: both sections and effective version.
If expansion denied or missing: review, not approve.Performance and operating cost
Smaller chunks improve search specificity but can raise index size and require expansion calls. Larger chunks carry more context but increase model tokens and may bury the key exception. Evaluate chunk sizes on a fixed set of question-document pairs and measure whether the complete governing rule reaches the model. For M passages retrieved, expansion is O(M) lookups before deduplication. Preserve source locations so the final evidence IDs can be checked against the original document.
Common Mistakes
- Do not accept a relevant fragment as the complete rule.
- Do not let parent expansion bypass access controls.
- Do not strip the section ID when merging neighboring chunks.
Connected lessons
- Prompt Engineering
- Prompt patterns
- Retrieved context: select sufficient evidence before writing the answer
- Retrieved evidence: reconcile versions and conflicting facts
- Retrieval queries: resolve references without adding facts
- Numeric prompts: let code calculate and the model explain
- Project: measure a retrieval-backed answer gate
- Prompt evidence and output decisions
Continue with: Context distillation: shorten input without losing governing exceptions.
Continue with: Retrieval prompts: pack evidence and map claims.
prompt engineering
tutorial