Project: review a single-branch policy and its transfer

Frame two decisions

The first question is whether the July routing policy changed North’s breach rate through September relative to a credible untreated path. The second is whether leadership should expect the same effect in a later group of branches. These are not interchangeable. Record the local unit, launch date, rate numerator and denominator, target branches and intended rollout date before touching weights. The panel clock supplies the raw timing contract.

Assemble the local design

Create an eligibility ledger for every potential donor: treatment status, shared-queue exposure, measurement changes and missing months. Freeze donors and fitting dates. Fit nonnegative weights that sum to one using pre-policy outcomes only, then store pre-fit error, monthly post gaps and an average gap. Show the untreated donor trajectories beside the treated branch. A convincing-looking post gap with weak pre-fit remains an unresolved design, not a success. The donor gate and fit lesson define the checks.

Run falsification and sensitivity checks

Refit placebo units using their own eligible pools, disclose the pre-fit threshold and compare accepted fit ratios. Rerun declared donor exclusions and fitting windows, storing both fit and gap for every result. Keep a log of branches excluded because they share queues with North. A sensitivity range is not an interval estimate. A placebo rank is not a design-based p-value when the routing policy was not randomly assigned. Placebos and reruns answer different questions.

Create a separate transfer file

List the target population’s pre-policy branch mix and the attributes that might alter the policy effect. A single North estimate does not populate every target stratum. Mark the transfer estimate unavailable until defensible effects exist for each target group, or produce a labeled scenario with a stated assumed range. Never present the local synthetic-control gap as a guaranteed network-wide benefit. Standardization applies only after that support exists.

Write a decision note with stop conditions

The executable gate below confirms that the required artifacts exist and that target strata have effect support. It does not infer causality from booleans. The review note should state whether the local effect is supported, which assumptions remain contestable, and whether transfer is currently estimable. If the donor pool collapses, pre-fit is unacceptable or the target mix includes an unsupported stratum, stop at a descriptive or scenario result. The broader rollout project covers multiple adoption cohorts.

Implementation

def policy_transfer_gate(local_artifacts, target_share, effect_by_stratum):
    required = {"donor_ledger", "frozen_pre_window", "pre_rmspe",
                "monthly_post_gaps", "placebo_log", "sensitivity_log"}
    missing = sorted(required - set(local_artifacts))
    if missing:
        raise ValueError("missing local artifacts: " + ", ".join(missing))
    if local_artifacts["pre_rmspe"] < 0 or not local_artifacts["monthly_post_gaps"]:
        raise ValueError("invalid local fit record")
    if abs(sum(target_share.values()) - 1) > 1e-12:
        raise ValueError("invalid target composition")
    unsupported = sorted(stratum for stratum, share in target_share.items()
                         if share > 0 and stratum not in effect_by_stratum)
    return {"local_packet_complete": True,
            "transfer_estimable": not unsupported,
            "unsupported_target_strata": unsupported}

packet = {"donor_ledger": ["West", "Harbor"], "frozen_pre_window": [1, 6],
          "pre_rmspe": .008, "monthly_post_gaps": [-.05, -.06, -.04],
          "placebo_log": ["West", "Harbor"],
          "sensitivity_log": ["drop_west", "later_start"]}
review = policy_transfer_gate(packet, {"urban": .3, "regional": .7},
                              {"urban": -.05})
assert review == {"local_packet_complete": True,
                  "transfer_estimable": False,
                  "unsupported_target_strata": ["regional"]}

Performance and operating cost

The gate costs O(A + S) expected time and O(S) output space for A artifact keys and S target strata. Running placebo refits and sensitivity designs is much more expensive. A complete packet only makes the evidence reviewable; it cannot turn unsupported transfer into an estimate.

Common Mistakes

Do not equate a complete evidence packet with causal proof.
Do not quote a population effect when required target strata have no supported estimates.
Do not conceal donor exclusions, failed pre-fit or unfavorable reruns.