Branch-cluster bootstrap for a rollout contrast

Last updated: 5 Oct 20265 min read

tutorial

A branch-cluster bootstrap resamples entire branches, preserving each branch’s repeated monthly outcomes while quantifying sampling variation in a specified rollout contrast.

Use the assignment unit

The triage screen is assigned to branches, not individual cases or months. Outcomes from the same branch share staff, demand and management decisions across time. Resampling branch-month rows independently breaks that dependence and can make an effect look more precise than it is. The code starts from one pre-to-post change per branch, then resamples treated branches and clean comparison branches separately. The cohort-time contrast defines which two months enter.

Keep design groups intact

A treated branch remains treated in every resample draw; a control branch remains a control. Resampling within groups maintains the designed group sizes for a simple conditional teaching interval. Each draw averages treated changes, averages control changes, and subtracts them. The fixture has more than one branch per group so variation is visible, although six branches are far too few for confident coverage claims.

Read the interval narrowly

The percentile endpoints describe variation under this case of branch sampling and frozen pre/post changes. They do not incorporate uncertainty about whether parallel trends holds, whether exposure spilled across branches, or whether adoption was assigned for an unmeasured reason. If a policy was assigned to regions containing several branches, the independent unit may be the region rather than the branch. A branch bootstrap cannot repair a design with only one independent treated region.

Beware few-cluster inference

Standard cluster-adjusted intervals and simple bootstrap percentiles can perform badly with very few treated clusters. Report the number of treated and comparison clusters prominently and seek an inference method suited to that design. Do not make a significance claim merely because a teaching interval excludes zero. Predefine the target effect, comparison set and inference plan before inspecting the outcome. Pre-trend checks are separate design evidence.

Preserve the full pipeline when necessary

This routine conditions on calculated branch changes. If the released estimate uses propensity weighting, cohort-time aggregation or sample exclusions learned from data, a full uncertainty analysis must repeat the appropriate steps inside each resample. Document whether interval variability covers only the final contrast or the entire estimation pipeline. The audit project records that boundary.

Implementation

import random

def branch_change_interval(treated_changes, control_changes,
                           repetitions=600, seed=71):
    if len(treated_changes) < 2 or len(control_changes) < 2 or repetitions < 100:
        raise ValueError("insufficient clusters or resamples")
    def contrast(treated, control):
        return sum(treated) / len(treated) - sum(control) / len(control)
    generator = random.Random(seed)
    estimates = sorted(contrast(
        generator.choices(treated_changes, k=len(treated_changes)),
        generator.choices(control_changes, k=len(control_changes)))
        for _ in range(repetitions))
    return contrast(treated_changes, control_changes), (
        estimates[int(.025 * repetitions)], estimates[int(.975 * repetitions)])

treated_branch_changes = [-.06, -.04, -.07]
control_branch_changes = [.01, .00, .02]
effect, interval = branch_change_interval(
    treated_branch_changes, control_branch_changes)
assert abs(effect - (-.06666666666666667)) < 1e-12
assert interval[0] <= effect <= interval[1]

Performance and operating cost

For B resamples and C branches, direct recomputation costs O(B × C + B log B) time and O(B + C) space. Resampling full branch trajectories adds their monthly rows to each draw. Computational speed does not solve a small number of independent assignment units.

Common Mistakes

Do not resample branch-month rows independently.
Do not imply that a bootstrap interval validates parallel trends.
Do not use a many-cluster approximation when only a few treated clusters exist.