Project: audit a staggered support-branch rollout

Freeze the branch-month contract

Take the monthly service-breach rate among cases created at each branch as the outcome. Save numerator, denominator, month, branch ID and extraction cutoff, and keep the first actual-use month from a separate rollout log. Define how partial launch months, branch closures and missing reports are handled. The panel audit checks unique rows and exposure timing before effects are calculated.

Estimate supported cohort-time comparisons

For each adoption cohort and later month with enough follow-up, use the last pre-adoption month as a baseline and compare with branches not yet treated at the later month. Store which branches entered each comparison and why others were excluded. Do not pool every cohort-time cell before examining its support and business meaning. The contrast lesson supplies an inspectable arithmetic check.

Challenge the design

Plot explicit pre-period contrasts by adoption cohort and show their uncertainty and branch counts. Record any rollout anticipation, staffing changes, case-routing changes or metric-definition changes. Build a pre-rollout shared-queue map; exclude indirectly exposed controls in the direct-effect analysis and report how many remain. Event-time checks and spillover screens are necessary but not proofs.

Report branch-level uncertainty

Resample or otherwise account for repeated outcomes at the assignment unit. If only a few independent treated regions exist, obtain an inference design suited to that fact rather than relying on a routine branch bootstrap. Show both the effect estimate and the number of independent treated and comparison units. The bootstrap lesson is a teaching calculation for adequate independent clusters.

Make the conclusion conditional

The review packet names the target branch cohort, comparison dates, treatment definition, parallel-trend rationale, spillover rule, uncertainty method and all unsupported cohort-time cells. The compact gate below catches missing rollout support and missing panel rows. It cannot establish a causal claim by itself. If support or assumptions fail, report descriptive changes instead of an effect estimate.

Implementation

def rollout_review_gate(panel, first_use_month, pre_month,
                            post_month, treated_cohort):
    if not pre_month < treated_cohort <= post_month:
        raise ValueError("invalid study periods")
    treated = {branch for branch, month in first_use_month.items()
               if month == treated_cohort}
    controls = {branch for branch, month in first_use_month.items()
                if month is None or month > post_month}
    if not treated or not controls:
        raise ValueError("no cohort or clean comparison")
    missing = [(branch, month) for branch in treated | controls
               for month in (pre_month, post_month)
               if (branch, month) not in panel]
    if missing:
        raise ValueError("missing required branch-month")
    def mean_change(branches):
        return sum(panel[branch, post_month] - panel[branch, pre_month]
                   for branch in branches) / len(branches)
    return {"treated_branches": len(treated),
            "comparison_branches": len(controls),
            "descriptive_contrast": mean_change(treated) - mean_change(controls)}

adoption = {"North": 3, "East": 3, "West": None, "South": 5}
breaches = {("North", 2): .21, ("North", 4): .16,
            ("East", 2): .19, ("East", 4): .15,
            ("West", 2): .17, ("West", 4): .18,
            ("South", 2): .14, ("South", 4): .15}
review = rollout_review_gate(breaches, adoption, 2, 4, 3)
assert review["treated_branches"] == 2
assert review["comparison_branches"] == 2
assert abs(review["descriptive_contrast"] - (-.055)) < 1e-12

Performance and operating cost

For B branches, a complete dictionary-backed gate costs O(B) expected time and O(B) space for group and missing-key sets. Full estimation, event-time plots and branch-level inference cost more. No computational check can create a counterfactual when the comparison design is unsupported.

Common Mistakes

Do not report a causal effect merely because the descriptive contrast is nonzero.
Do not drop exposed comparison branches without reporting the resulting population.
Do not hide missing branch-months or a weak cluster count.