Project: extract reviewable action frames from incidents

Last updated: 6 Oct 20265 min read

project

Stage who did what to which system, with negation and evidence, before an action appears in an incident timeline.

Choose the operational question

Responders need to distinguish an attempted rollback from a completed rollback. The extractor proposes action frames from notes; a reviewer confirms critical ones before they enter the timeline. Each frame includes actor, predicate, target, polarity, modality and source offsets. It does not authorize a production action. Preserve the note revision so a corrected sentence can invalidate its old frame.

Build hard examples

Annotate passive voice, quoted plans, negated retries, simultaneous actions and pronouns that refer to earlier services. Include similar deployments from separate incidents. Group all notes from one incident before splitting. The frame contract defines the labels; the decoder checks that extracted arguments are grounded.

Compare and stage

Start with a dependency-based baseline and compare a learned extractor on the same audit. Score full frames and costly polarity mistakes, not only token spans. Show uncertain actor links and conflicting frames to a reviewer with surrounding text. Link approved frames to timeline relations only after their source revision and identity scope are checked.

Operate safely

Version tokenizer, parser, role labels and catalog snapshot together. Monitor missed rollbacks, false completed actions, reviewer corrections and stale frames after note edits. Delete derived frames when a note is removed under retention policy. If a new model increases wrong-polarity frames, revert the model bundle while retaining the reviewer audit.

Implementation

def stage_action_frame(frame, note_revision):
    if frame["source_revision"] != note_revision:
        return {"state": "stale", "reason": "note-revised"}
    if frame["polarity"] != "affirmed" or frame["modality"] != "actual":
        return {"state": "review", "reason": "not-confirmed-action"}
    return {"state": "review", "reason": "confirm-operational-action"}

candidate = {"source_revision": "r47", "polarity": "negated",
             "modality": "actual", "predicate": "retry"}
assert stage_action_frame(candidate, "r47")["reason"] == "not-confirmed-action"

Performance and operating cost

The gate is O(1) time per frame; extracting and reviewing action spans costs more. A conservative review queue may handle more frames, but it prevents a denied or merely planned action from being entered as completed. Measure incident-level false action rate alongside extraction throughput.

Common Mistakes

Writing an extracted action directly into an incident record.
Ignoring negation or modality when building a timeline.
Keeping frames tied to an edited note revision.
Joining similar actions from unrelated incidents.