Annotation prompts: minimize private input while preserving context

Last updated: 5 Oct 202611 min read

tutorial

Annotation input should contain a stable item ID, scoped text, permitted metadata, and a record of redaction. Names, addresses, payment details, and unrelated messages rarely help classify a requested resolution. Strip those fields before model-assisted drafting or reviewer export. Context still matters: a phrase such as 'send another' may refer to a replacement only when a preceding message identifies the product and request. Mark missing context rather than letting the model fabricate it.

Operational case

Three Meridian disagreements come from excerpts missing earlier thread turns. The reviewer retrieves only the preceding relevant message through an approved view, preserving redaction. Two excerpts now have clear requested outcomes, while one stays unknown after context review. The audit ledger stores the excerpt ID and context receipt rather than copying a full private conversation into the prompt. The other four disagreements concern ambiguous wording, not missing data.

item: M-17
excerpt: 'Could you send another?'
redaction: customer identity removed
context_request: preceding product/resolution message only
status: UNKNOWN until scoped context is reviewed

Performance and review cost

Redacting B input bytes is O(B) processing; retrieving the few needed messages adds I/O proportional to those messages, not the whole archive. Narrow context lowers exposure and annotation noise. Keep the redaction version so reviewers can distinguish a truly absent fact from one intentionally withheld.

Common Mistakes

Do not paste full ticket histories by default.
Do not infer missing context from phrasing alone.
Do not erase the fact that a field was redacted.

Connected lessons

prompt engineering

annotation quality