Project: release receipt triage with lineage, canary checks and rollback

Produce a reviewable candidate

Train or reuse a receipt-triage model from a pinned input snapshot. Write a manifest naming code revision, feature schema, class order, evaluation population and artifact digest. Deliberately remove one field and show that promotion refuses the candidate. Artifact lineage] is the first acceptance criterion.

Prove feature and release behavior

Recompute sampled offline features at their original decision times and compare them with serving values. Stage the candidate as shadow traffic so its scores cannot perform actions. Then route a stable small customer slice through canary assignment. Inject one candidate timeout and verify fallback plus a recorded incident. Rollout controls] define this behavior.

Monitor without invented outcomes

Track request count, errors, latency, abstention and feature freshness immediately. Calculate quality only after labels mature under a fixed cutoff, and state the evaluated denominator. Simulate a source parser defect and show why the response is repair rather than retraining. Monitoring] should separate those causes.

Demonstrate rollback

Keep the prior artifact and preprocessing manifest available. Switch the serving pointer back after a guardrail breach, then run known-good requests to verify previous outputs. Submit the manifest, gates, split IDs, parity report, canary assignment tests, latency sample and rollback transcript. A diagram without failure-case output is incomplete.

Implementation

def release_gate(candidate, parity_mismatches, smoke_passed, prior):
    if parity_mismatches:
        raise ValueError("feature parity failed")
    if not smoke_passed or not candidate.get("artifact_sha256"):
        raise ValueError("candidate evidence incomplete")
    if candidate["urgent_recall"] < prior["urgent_recall"]:
        raise ValueError("urgent-class guardrail failed")
    return candidate["artifact_sha256"]

Performance and operating cost

Shadow traffic adds model execution per mirrored request; canary monitoring adds storage for bounded decision logs. The full project cost includes replay, evaluation and retained rollback artifacts.

Common Mistakes

Do not let a shadow score trigger a real receipt action.
Do not calculate quality before outcomes mature.
Do not delete the prior artifact before rollback is tested.