Promotion evidence: bind evaluation, contract and rollback to one digest

Last updated: 7 Oct 20265 min read

tutorial

A release decision is valid only for the exact artifact and environment tested; moving an alias must retain a known rollback target.

Describe what the evidence actually covers

An evaluation record should name model digest, data snapshot, feature contract, metric code revision, slice definitions and hardware profile. A passing quality score from another digest is not evidence for the candidate being deployed. Keep the builder identity and dependency lock alongside the package. The record is an attestation about a build and evaluation; its trust depends on who can issue and alter it. Artifact verification checks that serving receives the attested bytes.

Make promotion a small, auditable transition

Separate candidate registration from the serving pointer. Before moving the pointer, verify artifact digest, contract compatibility, evaluation limits, privacy approval where relevant and serving latency. Resolve the current pointer to an immutable rollback digest and confirm the old package is still loadable. A mutable alias is useful operationally, but the change event must store old and new digests so an incident can reconstruct what ran. Registry gates provide the decision context.

Design rollback before canary

Rollback is more than resetting an alias if the feature producer, tokenizer or schema changed. Keep the old serving image, compatible inputs and capacity until the candidate clears its observation window. Stage a rollback drill using the same deployment method used in production, then measure time to restore customer outcomes. Canary controls should halt on tail latency, error and quality guardrails rather than only average accuracy.

Reject stale or borrowed evidence

A digest may pass yesterday and fail today because a dependency image, policy or input contract changed. Define evidence freshness and environment scope. Reject a signed report from a different project or a report whose artifact digest does not match the candidate. The release project tests a swapped report, missing rollback package and broken input contract.

Implementation

def promotion_ready(candidate, evidence, rollback):
    required = {"artifact_digest", "feature_contract", "data_snapshot"}
    if not required <= evidence.keys():
        return {"state": "hold", "reason": "missing-evidence"}
    if evidence["artifact_digest"] != candidate["artifact_digest"]:
        return {"state": "hold", "reason": "digest-mismatch"}
    if evidence["feature_contract"] != candidate["feature_contract"]:
        return {"state": "hold", "reason": "contract-mismatch"}
    if not evidence.get("quality_passed") or not evidence.get("latency_passed"):
        return {"state": "hold", "reason": "failed-gate"}
    if not rollback.get("loadable") or not rollback.get("input_compatible"):
        return {"state": "hold", "reason": "no-rollback"}
    return {"state": "ready", "rollback_digest": rollback["artifact_digest"]}

candidate = {"artifact_digest": "sha256:r8", "feature_contract": "receipt-v3"}
evidence = {**candidate, "data_snapshot": "snapshot-82",
            "quality_passed": True, "latency_passed": True}
old = {"artifact_digest": "sha256:r7", "loadable": True,
       "input_compatible": True}
assert promotion_ready(candidate, evidence, old)["state"] == "ready"
assert promotion_ready(candidate, {**evidence, "artifact_digest": "sha256:other"},
                       old)["state"] == "hold"

Performance and operating cost

Comparing a fixed release record is O(1) time and space. The expensive work is producing trustworthy evidence: evaluation runs, load tests and rollback drills. The code checks key equality and gate booleans, not cryptographic signatures or issuer authorization; enforce those at the release system boundary.

Common Mistakes

Applying a passing report to a different artifact digest.
Keeping a rollback alias but deleting its underlying package.
Moving the model pointer while the old input contract is no longer emitted.
Treating a signed report from an untrusted issuer as sufficient evidence.