Spillover-aware comparison pools for branch rollouts

Last updated: 5 Oct 20265 min read

tutorial

A branch that has not received an intervention may still be exposed through shared staff, customers or routing, so untreated status alone may not make it a clean control.

Map the possible pathways

A new triage screen at North may route overflow to West, train agents who later cover West, or change customer behavior across both. A West branch-month marked untreated can then have a changed breach rate. The no-interference assumption behind a simple branch comparison is doubtful. Before estimating, map shared queues, staffing rotations and service regions using records from before rollout. A causal graph helps separate baseline relationships from effects of the rollout.

Construct an exposure rule

The small routine below treats a branch as indirectly exposed in a month if any listed neighbor has already adopted. It excludes those branches from the untreated comparison pool while keeping never-treated, unexposed branches. In the fixture, West is adjacent to a treated North branch and is excluded; South is remote and remains eligible. This is a screening rule, not proof that all spillover paths were captured.

Do not erase the population silently

Removing exposed controls changes the population represented by the estimate. If most branches share one routing network, there may be no clean controls after rollout. Report counts by direct treatment, indirect exposure and neither at each month. Consider a different estimand, such as the total network effect, or a design with more independent networks. A narrower clean comparison can be more honest than a precise-looking estimate from contaminated controls.

Avoid post-treatment geography

A customer network rebuilt from traffic after triage launched can already reflect the intervention. Defining neighbors from that network may select controls using post-treatment behavior. Freeze the exposure map using pre-rollout operations and then test sensitivity to plausible wider connections. Shared staffing logs after rollout can be used to audit actual contamination, but should not be quietly used to redefine the original design without documenting the change.

Check indirect outcomes separately

If West receives overflow, its breach rate may rise. That is an effect of North’s rollout, not evidence that the triage screen failed within North. A direct-effect analysis that excludes West and a separate spillover analysis of West answer different questions. Cohort-time contrasts should name which one they estimate. Once exposure is network-wide, branch-level confidence intervals may miss correlated shocks; cluster inference needs the correct independent unit.

Implementation

def clean_controls(first_use_month, adjacency, evaluation_month):
    if evaluation_month <= 0:
        raise ValueError("invalid evaluation month")
    branches = set(first_use_month)
    if set(adjacency) != branches:
        raise ValueError("incomplete adjacency map")
    for branch, neighbors in adjacency.items():
        if not set(neighbors) <= branches or branch in neighbors:
            raise ValueError("invalid neighbor list")
    directly_treated = {
        branch for branch, first_month in first_use_month.items()
        if first_month is not None and first_month <= evaluation_month}
    indirectly_exposed = {
        branch for branch, neighbors in adjacency.items()
        if branch not in directly_treated and set(neighbors) & directly_treated}
    return branches - directly_treated - indirectly_exposed, indirectly_exposed

first_use = {"North": 3, "West": None, "South": None}
neighbors = {"North": ["West"], "West": ["North"], "South": []}
controls, spillover = clean_controls(first_use, neighbors, 4)
assert controls == {"South"}
assert spillover == {"West"}

Performance and operating cost

For B branches and E listed neighbor links, the screening rule is O(B + E) expected time and O(B + E) space including input adjacency. Building a defensible exposure map and measuring hidden shared queues are the substantive costs.

Common Mistakes

Do not call a branch unexposed only because it lacks the new screen.
Do not define control neighbors from a network created after rollout.
Do not keep a precise estimate when the clean comparison pool is empty.