Treatment-effect bounds under differential attrition

Last updated: 5 Oct 20265 min read

tutorial

With a valid assignment design and binary outcomes missing in each arm, an effect interval subtracts the largest possible control rate from the smallest possible treated rate, and vice versa.

Define the direction of the effect

Let the outcome be a deadline breach, where lower is better. The effect is assigned-policy breach probability minus control breach probability. A negative number favors the policy. Among 50 assigned-policy tickets, 40 outcomes are known and 15 breached. Among 50 control tickets, 45 outcomes are known and 20 breached. The missing counts differ, so comparing 15/40 with 20/45 conditions on two selected responder sets. The rate-bounds lesson keeps both full denominators.

Combine group endpoints correctly

The treated rate lies from 0.30 to 0.50. The control rate lies from 0.40 to 0.50. Therefore the treated-minus-control effect lies from -0.20 to +0.10. The lower effect pairs the smallest treated rate with the largest control rate; the upper pairs the largest treated rate with the smallest control rate. Using lower minus lower would omit feasible outcomes and understate uncertainty. The code computes and checks this ordering.

Keep assignment and interference assumptions visible

The difference of group risks can represent an assignment effect only when assignment supports exchangeability for the eligible units, treatment is defined consistently and spillovers are handled. These missing-outcome bounds do not repair confounding in a management-selected rollout. If the trial assigns branches, not tickets, sampling uncertainty belongs at the branch level even though the arithmetic uses ticket counts. Assignment-aware inference treats that unit distinction.

Interpret the zero crossing

Because the interval contains zero, the observed outcomes and binary support alone do not determine whether the policy helped. This is not a null-hypothesis result or a p-value. A narrow conventional confidence interval fitted only to observed tickets would answer a different, stronger-assumption question. Show both the endpoint counts and the interval so reviewers can see why the sign remains unresolved.

Choose a defensible next constraint

Operational evidence might bound breach risk among missing tickets instead of allowing the entire zero-to-one range. Such a restriction narrows the interval only to the extent it is credible. The scenario lesson calculates that narrower range, while the decision lesson translates an interval into a policy action.

Implementation

def arm_rate_bounds(assigned, observed, breaches):
    if assigned <= 0 or not 0 <= breaches <= observed <= assigned:
        raise ValueError("inconsistent arm counts")
    missing = assigned - observed
    return breaches / assigned, (breaches + missing) / assigned

def effect_bounds(treated_counts, control_counts):
    treated_low, treated_high = arm_rate_bounds(*treated_counts)
    control_low, control_high = arm_rate_bounds(*control_counts)
    return treated_low - control_high, treated_high - control_low

treated = (50, 40, 15)
control = (50, 45, 20)
lower_effect, upper_effect = effect_bounds(treated, control)
assert abs(lower_effect - (-.20)) < 1e-12
assert abs(upper_effect - .10) < 1e-12

Performance and operating cost

Given arm counts, interval arithmetic is O(1) time and space. Validating unit-level assignment, missingness and outcome status requires a scan of the underlying study frame; dependence-aware sampling uncertainty is a separate analysis.

Common Mistakes

Do not subtract matching endpoints when bounding a difference.
Do not use ticket-level precision when branches were assigned.
Do not call an effect interval causal if treatment groups were self-selected.