Project: promote a receipt model with artifact and rollback evidence

Last updated: 6 Oct 20265 min read

project

Build a release gate that ties package bytes, feature contract, evaluation and rollback to one candidate digest.

Define the release bundle

A new receipt-risk model contains weights, a feature map and a metadata manifest. Pin their digests and record the build identity, source revision, data snapshot, evaluation slices and feature contract. The candidate remains registered but unserved until all checks pass. The artifact boundary rejects missing or altered files before any loader runs.

Create adversarial release cases

Prepare one complete package, one altered weight file, one borrowed evaluation report, one missing tokenizer or feature map, and one candidate that cannot be rolled back because the previous contract is no longer emitted. Freeze expected hold reasons. Include a quality report with good overall accuracy but failed urgent-case recall; its gate must fail. Promotion evidence must refer to the exact digest and environment under review.

Stage and observe the transition

Verify digests, evaluate quality and serving cost, confirm the old package loads, then move a staging pointer. Run shadow and canary traffic while recording old and new digests in every release event. Keep a human-readable decision and the machine-check results. Avoid loading arbitrary serialized objects from a package merely because its registry entry exists. Canary behavior tests customer outcomes after the pointer moves.

Exercise recovery

Tamper with the candidate after staging and confirm deployment rejects it. Simulate a feature-producer change during canary; roll back model and producer together if required. Report verification time, promotion holds, canary outcomes and recovery duration. The final artifact is a release record that identifies exactly what ran, why it was accepted and which package can replace it during an incident.

Implementation

def release_state(artifact_check, evidence_check, rollback_check,
                  canary_error_rate):
    for name, passed in (("artifact", artifact_check),
                         ("evidence", evidence_check),
                         ("rollback", rollback_check)):
        if not passed:
            return {"state": "hold", "reason": name}
    if not 0 <= canary_error_rate <= 1:
        raise ValueError("error rate must be a fraction")
    if canary_error_rate > 0.012:
        return {"state": "rollback", "reason": "canary-errors"}
    return {"state": "promote"}

assert release_state(True, True, True, 0.008)["state"] == "promote"
assert release_state(False, True, True, 0.008)["state"] == "hold"
assert release_state(True, True, True, 0.021)["state"] == "rollback"

Performance and operating cost

The final decision uses O(1) time and space; artifact hashing costs O(B) for B package bytes and canary observation costs requests and retained aggregates. A small canary may be too noisy to support a tight error threshold, so set its sample size and observation period before looking at results.

Common Mistakes

Executing a model loader before verifying package identity.
Accepting evaluation evidence from a different digest.
Declaring rollback ready without loading the old package.
Ignoring a failed canary because the offline score improved.