Project: release a versioned receipt feature admission gate

Last updated: 6 Oct 20265 min read

project

Build a replayable admission gate for a receipt model, then prove a unit migration does not break the candidate or rollback service.

State the operating task

A payment intake system scores receipts with amount, merchant age and event time. The model must never see an amount with unknown units, a future event or a field silently converted from text. Define accepted, rejected and fallback outcomes, plus one short reason code for each failure. The feature contract is part of the model release package, not an informal convention in the request handler.

Build adversarial fixtures

Create valid records, null merchant ages, boolean amounts, oversized amounts, stale timestamps, future timestamps and an upstream change from cents to thousands of cents. Freeze expected decisions before writing migration code. Include both the new candidate and the old rollback model as consumers. Schema evolution forces the migration to support both until rollback is no longer needed.

Wire admission to promotion

Store the contract version, model digest and source schema revision with each test result. Reject a candidate whose fixture pass rate falls below its declared threshold, even if offline quality improved. Shadow live requests through the new gate and compare outcome counts with the old gate without using the shadow output for customer decisions. Keep restricted examples of rejected data for debugging; publish only reason counts and low-cardinality source labels. Shadow rollout provides the comparison pattern.

Review the release evidence

Report fixture failures, live rejection rate, unknown-version rate, candidate and rollback compatibility, and median and tail admission latency. Simulate an upstream unit change after candidate promotion; the gate must hold requests or use an approved fallback, not issue scores in the wrong units. Record who approves retirement of the old field and the earliest date the rollback model can be removed.

Implementation

def admission_release(candidate_checks, rollback_checks, shadow_counts):
    if not all(candidate_checks.values()):
        return {"state": "hold", "reason": "candidate-fixture"}
    if not all(rollback_checks.values()):
        return {"state": "hold", "reason": "rollback-fixture"}
    total = shadow_counts["accepted"] + shadow_counts["rejected"]
    if total < 470:
        return {"state": "hold", "reason": "small-shadow-sample"}
    if shadow_counts["rejected"] / total > 0.03:
        return {"state": "hold", "reason": "rejection-rate"}
    return {"state": "ready", "checked": total}

fixtures = {"units": True, "future_clock": True, "nulls": True}
rollback = {"old_amount_minor": True}
assert admission_release(fixtures, rollback,
                         {"accepted": 489, "rejected": 11})["state"] == "ready"
assert admission_release(fixtures, {"old_amount_minor": False},
                         {"accepted": 489, "rejected": 11})["state"] == "hold"

Performance and operating cost

The gate evaluation is O(f) time for fixture results and O(1) extra space aside from its response. Running both model contracts in shadow doubles validation work for sampled requests. Set a bounded review queue for rejected records, and treat threshold selection as an explicit product decision grounded in observed traffic.

Common Mistakes

Passing candidate fixtures while ignoring the retained rollback model.
Treating a high rejection rate as harmless because accepted predictions look accurate.
Using shadow outputs for customer decisions.
Publishing raw rejected receipts through a monitoring dashboard.