Project: coordinate a field-scanner federated training round

Last updated: 7 Oct 20265 min read

project

Pin a scanner model, validate client updates, abort a sparse round and release only an independently evaluated aggregate.

Prepare the fleet and manifest

Field scanners classify damaged shipping labels locally. Raw photos remain on-device under the product policy. Freeze a global model digest, local training settings, tensor shape, example-count cap and round deadline. Define eligibility by app version, approved network and power state, then record eligibility by warehouse region before sampling. The set that can participate is not automatically representative of the entire fleet. The round manifest makes every accepted update traceable to one base.

Inject failures

Invite 470 devices across four regions. Some return duplicate update IDs, some train against the prior base and a remote region loses connectivity. Reject stale and malformed envelopes; do not retry them as if they were new training results. The first round misses the minimum accepted-client and regional-coverage gates, so abort it without creating a promotable model. Participation accounting distinguishes invitations, completed uploads and accepted updates.

Run a controlled second round

Start a new round with a fresh deadline and the same pinned base, rather than extending the completed round invisibly. Recompute eligibility and ensure the remote region has a valid opportunity to participate. Cap contribution weight per client and record the exact accepted update IDs and aggregate recipe. Compare the candidate on a held-out labeled set that does not consist only of the most connected sites. A claimed privacy guarantee requires a separately validated protocol; this exercise makes no such claim.

Stage the artifact

Package the candidate with runtime and preprocessing digests, evaluate rare damage types and scan-quality slices, then stage a small device cohort. Monitor local failure, offline reporting coverage and rollback events before broad rollout. Publish the round history, rejected-update counts, participation by region and evaluation limits. Edge release controls manage installation; a training-round pass alone cannot guarantee device-side behavior.

Implementation

def scanner_round_gate(accepted, minimum_clients, region_counts, minimum_region):
    if accepted < 0 or minimum_clients <= 0 or minimum_region < 0:
        raise ValueError("invalid policy or counts")
    if accepted < minimum_clients:
        return "abort:client-floor"
    if not region_counts or any(count < minimum_region
                                for count in region_counts.values()):
        return "abort:region-coverage"
    if sum(region_counts.values()) != accepted:
        return "hold:accounting-mismatch"
    return "evaluate:aggregate"

regions = {"north": 82, "south": 71, "east": 64, "west": 39}
assert scanner_round_gate(256, 220, regions, 47) == "abort:region-coverage"
assert scanner_round_gate(256, 220, {**regions, "west": 47}, 47) ==        "hold:accounting-mismatch"
assert scanner_round_gate(256, 220, regions, 39) == "evaluate:aggregate"

Performance and operating cost

The gate is O(r) time and O(1) extra space for r regions. Two rounds increase device battery, network and coordination cost; aborting a sparse first round prevents an unrepresentative aggregate from being treated as production evidence. A later rollout still incurs package downloads and delayed offline telemetry.

Common Mistakes

Reusing an old-base update in a later round.
Promoting a round that passed global participation but missed one region.
Presenting local-data storage as a complete privacy proof.
Skipping independent evaluation because aggregate training loss fell.