Project: review a small branch rollout with spillovers

Reconstruct the rollout decision

Start with branch-month breach numerator and denominator, first-use date, eligible branch list and the actual rule that selected rollout branches. If a lottery selected two of five eligible branches, preserve its assignment restrictions. If management selected branches by need, record that instead and do not present a uniform permutation test as exact inference. Assignment-aware testing has a narrow design requirement.

Map exposure before looking at outcomes

Build a pre-rollout network from shared queues and staffing. Classify every branch-month as directly treated, indirectly exposed or unexposed. Report uncertain connections and how a broader map changes group counts. Estimate a direct contrast using clean controls and a separate indirect contrast among untreated branches only. Exposure classes and the spillover contrast answer different questions.

Test whether one branch drives the story

Recompute the primary contrast after omitting each assignment unit, retain all planned negative controls and pre-rollout placebo windows, and show the treated and comparison branch counts. A leave-one-out range is a diagnostic, not a confidence interval. A failed negative control is a reason to investigate, not a command to delete that metric. Influence checks and falsification checks belong beside the main result.

State the inference limit

When a genuine restricted lottery exists, calculate the randomization distribution under its sharp no-effect null. With observational selection or few independent regions, use an inference method suited to the assignment structure and avoid automatic significance claims from ordinary branch bootstraps. Record the smallest attainable randomization tail probability where relevant. Cluster uncertainty does not fix a weak assignment design.

Make publication conditional

The release packet names the target population, periods, adoption groups, exposure rule, comparison support, trend evidence, falsification outcomes, branch influence and uncertainty scope. The gate below checks unique branches, disjoint exposure categories and at least one clean comparison. It does not prove causal identification. If the map leaves no unexposed branch, state that the direct comparison is unsupported.

Implementation

def small_rollout_gate(first_use_month, exposure_status, evaluation_month):
    if evaluation_month <= 0 or set(first_use_month) != set(exposure_status):
        raise ValueError("branch coverage mismatch")
    valid = {"direct", "indirect", "unexposed"}
    if any(status not in valid for status in exposure_status.values()):
        raise ValueError("unknown exposure status")
    for branch, first_use in first_use_month.items():
        adopted = first_use is not None and first_use <= evaluation_month
        if adopted != (exposure_status[branch] == "direct"):
            raise ValueError("exposure contradicts adoption month")
    direct = {branch for branch, status in exposure_status.items()
              if status == "direct"}
    indirect = {branch for branch, status in exposure_status.items()
                if status == "indirect"}
    unexposed = {branch for branch, status in exposure_status.items()
                 if status == "unexposed"}
    if not direct or not unexposed:
        raise ValueError("unsupported direct comparison")
    return {"direct": len(direct), "indirect": len(indirect),
            "unexposed": len(unexposed),
            "indirect_comparison_ready": bool(indirect)}

adoption = {"North": 4, "West": None, "South": None, "Harbor": None}
exposure = {"North": "direct", "West": "indirect",
            "South": "unexposed", "Harbor": "unexposed"}
review = small_rollout_gate(adoption, exposure, 5)
assert review == {"direct": 1, "indirect": 1,
                  "unexposed": 2, "indirect_comparison_ready": True}

Performance and operating cost

The gate scans B branches in O(B) expected time and O(B) space for classification sets. Full network reconstruction, trend comparison and design-specific inference add separate costs. A fast gate is not a causal approval certificate.

Common Mistakes

Do not represent a management-selected rollout as a randomized lottery.
Do not merge indirect and unexposed branches into one control label.
Do not publish a causal effect if the comparison set or assignment assumptions are unsupported.