Project: extract incident actions from validated syntax

Set the output contract

The incident team needs subject, action, object and observed status from notes such as a service restarting a queue. Define the difference between planned, attempted, completed and negated actions. Store character spans and source revision for all arguments. The output is a proposed action record, not an authoritative change log. Ambiguous pronouns and unknown service IDs go to review.

Prepare domain examples

Capture original incident notes with timestamps and speaker identity. Annotate entity spans, action cues, negation scope and reviewed service IDs. Include passive sentences, quoted commands, failed attempts, duplicated updates and later corrections. Group by incident and reserve recent incidents for audit. Validate parse heads and original offsets using the tree contract before feature extraction.

Compare paths with a baseline

Use local lexical and entity-type rules as a first candidate generator. Add syntactic paths only if the dependency route improves candidate recall or reduces false staged edges. Measure candidate recall, end-to-end exact action F1, false completed-action rate and review workload. Do not evaluate a relation classifier solely on gold entity pairs. Review examples where parsing fails but the lexical baseline succeeds.

Stage and revoke

Keep each proposed action with parser, tokenizer, entity model and rule versions. A reviewer approves, rejects or marks it superseded after a later note. Restrict evidence access to users who can read the incident. When the source changes, revalidate offsets and retire affected actions; never quietly update a previous approved record. The ledger project provides the state model for that handoff.

Implementation

def stage_incident_action(subject, verb, object_id, source_revision, status):
    allowed_statuses = {"planned", "attempted", "completed", "negated"}
    if status not in allowed_statuses or not all((subject, verb, object_id, source_revision)):
        raise ValueError("invalid incident action")
    return {"subject": subject, "verb": verb, "object": object_id,
            "source_revision": source_revision, "observed_status": status,
            "review_state": "staged"}

action = stage_incident_action("service-k", "restart", "queue-m", "note-47-v3", "attempted")
assert action["observed_status"] != "completed"

Performance and operating cost

The record gate is O(1) time and output space. Parsing every incident sentence and evaluating candidate pairs dominate compute; adjudication dominates human cost. Monitor p95 per-document latency, malformed parse rate, candidate recall and false completed actions together. A faster parser is not an improvement if it converts attempted or negated actions into approved completions.

Common Mistakes

• Equating “attempted to restart” with a completed restart.
• Omitting source revision from a staged action.
• Assuming all dependency parse outputs are structurally valid.
• Letting an approved action remain current after its source was corrected.

Read next

• Dependency trees: token identity, heads and structural checks
• Use dependency paths to propose, not assert, entity relations
• Project: build a reviewed entity-relation ledger from incident notes
• Coreference chains: track mentions without guessing identity
• Relation extraction with direction, negation and evidence