Edge model releases: pin runtime, preprocessing and cohort

Last updated: 7 Oct 20265 min read

tutorial

A device-side model is a release package with runtime and preprocessing requirements, not a model file copied to a handset.

Ship a complete decision package

A receipt-scanning handheld may classify image quality while offline. Its result depends on image resize rules, token or pixel normalization, runtime version, model bytes and route policy. Record those as one signed or otherwise authenticated manifest with digests and a minimum compatible app version. A new model paired with an old preprocessor is an untested release. Chain identity applies on-device as well as in a server, while artifact verification prevents loading unexpected bytes.

Assign devices before download

Choose rollout cohorts by stable device or site identity, not by each request. Stage first to test units, then a small production cohort, then broader fleets after health evidence arrives. Distinguish eligible, assigned, downloaded, installed and actually serving counts. A device that downloaded a package but never activated it is not exposed. Exposure evidence has the same assignment-versus-service distinction in online experiments; an edge fleet adds long offline intervals and varying app versions.

Check compatibility at activation

Download to a staging location, verify the manifest and bytes, run a local smoke input, then atomically change the active pointer. Retain the previous package until the new one has served safely. Reject activation when free storage, runtime capability or input shape is incompatible. A failing download should leave the current model working. Reversible promotion should name the exact prior package rather than an alias that can change while the device is offline.

Gate expansion on observed health

Compare crash rate, inference latency, local fallback, battery impact and missing-event coverage by cohort and device class. Low-connectivity devices may report late, so a quiet dashboard does not mean success. Hold expansion until the minimum observation window and reporting coverage are met. Offline telemetry supplies a bounded, delayed view of actual use. The handheld release project exercises a package that downloads successfully but cannot activate on an older runtime.

Implementation

def edge_activation(package, device):
    if package["app_min"] > device["app_version"]:
        return "hold:app-version"
    if package["runtime"] not in device["supported_runtimes"]:
        return "hold:runtime"
    if package["bytes"] > device["free_bytes"]:
        return "hold:storage"
    if not package["digest_verified"] or not package["smoke_passed"]:
        return "hold:package-check"
    return "activate"

package = {"app_min": 7, "runtime": "receipt-runtime-r3", "bytes": 47_000_000,
           "digest_verified": True, "smoke_passed": True}
device = {"app_version": 8, "supported_runtimes": {"receipt-runtime-r3"},
          "free_bytes": 82_000_000}
assert edge_activation(package, device) == "activate"
assert edge_activation(package, {**device, "app_version": 6}) == "hold:app-version"

Performance and operating cost

The activation gate is O(1) expected time and space for a small runtime set. Download and storage cost scale with package bytes, while smoke inference consumes device time and power. Cohort rollout reduces blast radius but prolongs mixed-version operation; maintain both versions only as long as compatibility and rollback require.

Common Mistakes

Counting a downloaded model as one that served decisions.
Updating the model without its preprocessing and runtime contract.
Replacing the current package before the new package passes a local smoke test.
Expanding a cohort while offline devices have not reported health.