Subgroup distributions and aggregation reversal

Last updated: 5 Oct 20265 min read

tutorial

An aggregate rate can move opposite to every subgroup rate when the mixture of groups changes.

Hold the question steady

Suppose an established route and a new route have different baseline success rates. In an earlier period, the established route succeeds on 90 of 100 deliveries and the new route on 1 of 10. In a later period, they succeed on 19 of 20 and 40 of 200. Each route improves—from 90% to 95%, and from 10% to 20%—yet the pooled rate falls from 91/110, about 82.7%, to 59/220, about 26.8% because volume shifts toward the harder route.

Report composition beside outcomes

Show subgroup numerator, denominator and rate for both periods, plus each group’s share of total volume. The pooled number is still a real operational measure; it answers a different question from within-route performance. To compare like-for-like performance, standardize both periods to a declared reference mix, then report the actual mix separately. Denominator discipline prevents a change in population from being mistaken for a process regression.

Avoid post-hoc slices

Choose route groups because they explain service conditions or allocation policy, not because one partition produces a preferred headline. Small cells carry large uncertainty; suppress or combine them under a documented rule. If group membership changes during the period, define whether a shipment belongs to its dispatch route, delivery route or current route. Aggregation grain must match that choice.

Test the reversal

Calculate the two within-route rates and the two pooled rates from the counts above. Then use the earlier route mix for both periods and compare standardized rates. A dashboard that displays only the pooled decline would miss the within-route improvements; a dashboard that displays only standardized gains would hide the operational burden of shifting volume.

Implementation

def rate(successes, attempts):
    if attempts <= 0 or not 0 <= successes <= attempts:
        raise ValueError("invalid counts")
    return successes / attempts

old_route_rates = (rate(90, 100), rate(1, 10))
new_route_rates = (rate(19, 20), rate(40, 200))
assert all(new > old for old, new in zip(old_route_rates, new_route_rates))
assert rate(59, 220) < rate(91, 110)

Performance and operating cost

Computing G subgroup counts from N records costs O(N) time and O(G) space. Standardization is O(G) once counts exist; uncertainty must still reflect the sampling or dependence structure.

Common Mistakes

Do not compare pooled rates without their group composition.
Do not choose subgroup boundaries after viewing the desired result.
Do not hide the actual operational volume mix when reporting standardized rates.