Project: publish evidence-backed incident handoff summaries

Last updated: 6 Oct 20265 min read

project

Turn an incident timeline into a short handoff that preserves confirmed facts, corrections, open questions and source evidence.

The handoff request

At shift change, the responder needs confirmed customer impact, current mitigation, owner and the next unresolved action. Set a word budget and a mandatory-field policy. Input can include chat excerpts, ticket updates and a runbook revision. Snapshot the accessible source set at generation time and record which responder may read it. A summary must not reveal a restricted note to someone who can see only the public incident log.

Prepare the timeline

Normalize event timestamps to one time base while preserving original strings, speaker IDs and document revisions. Mark explicit retractions and corrections. Select windows that include the latest confirmed state, not only the first high-similarity hits. Test on timelines with duplicated updates, contradictory cause guesses, a late rollback and an empty owner field. The previous runbook search project can retrieve useful context, but a search hit still needs permission and claim review.

Generate and verify

Draft a concise structured summary, then split it into claims. Require source offsets for every factual statement and manual review for unsupported, contradictory or missing mandatory fields. Check quantities and dates separately because tiny changes have large operational effects. Compare against an extractive baseline built from confirmed timeline sentences. Record factual support, required-field coverage, review burden and editing time. The evidence contract and claim review define the gates.

Release with a rollback path

Publish only a reviewed revision with a source snapshot digest. When an incident update invalidates the summary, mark it stale and generate a new draft; do not quietly mutate a signed-off handoff. The recipient should see freshness and unresolved questions. Restrict raw source snippets in logs, and retain the prior signed-off revision for audit. A model upgrade must replay the hard timeline set before it replaces the current bundle.

Implementation

from hashlib import sha256

def handoff_record(incident_id, source_revisions, claims, required_types):
    if not source_revisions or not required_types:
        raise ValueError("source revisions and required types are mandatory")
    if any(claim["status"] != "supported" for claim in claims):
        return {"incident_id": incident_id, "status": "manual-review"}
    observed = {claim["type"] for claim in claims}
    if not required_types <= observed:
        return {"incident_id": incident_id, "status": "manual-review"}
    manifest = "|".join(sorted(source_revisions)).encode("utf-8")
    return {"incident_id": incident_id, "status": "ready-for-signoff",
            "source_digest": sha256(manifest).hexdigest()}

handoff = handoff_record("incident-47", ["timeline-v4", "runbook-v7"],
                         [{"type": "impact", "status": "supported"}], {"impact"})
assert handoff["status"] == "ready-for-signoff"

Performance and operating cost

The local gate is O(c + r log r) time for c claims and r revision identifiers, with O(r) manifest space. Reading, selecting and generating over long timelines is far more expensive; measure p95 processing time and human signoff time separately. A source digest proves which revision IDs were declared, not that the content is faithful. Keep versioned source snapshots and a manual signoff trail for the actual factual check.

Common Mistakes

Treating “ready for signoff” as automatic publication.
Losing a later correction while shortening the source window.
Allowing a restricted runbook passage into a widely visible summary.
Updating a signed-off handoff without a new revision and review.