Stratified sampling and design weights for operational estimates

Last updated: 5 Oct 20265 min read

tutorial

A stratified sample draws within named groups; a population estimate must reflect each group’s population size and selection probability.

Choose strata before drawing

Divide the eligible support-case frame by intake channel and severity only when those fields are known for every case at selection time. Oversample a rare partner channel to study it well, but preserve its population count. A convenience queue is not a probability sample just because it contains several groups. Frame coverage must be checked before weighting can repair unequal selection.

Estimate at the right scale

Suppose the main channel has 1,200 cases, with 24 defects in a random sample of 120. The partner channel has 300 cases, with 24 defects in a random sample of 60. Their rates are 20% and 40%. The population-weighted rate is (1,200 × 0.20 + 300 × 0.40) / 1,500 = 24%. Pooling the 48 defects over 180 sampled cases gives 26.7% because the partner channel was oversampled.

Carry weights with records

Within a simple random sample from a stratum, the base weight is population count divided by sample count. Save stratum, sample design, selection probability and weight beside the reviewed outcome. Do not recompute a weight from the number of respondents if nonresponse changes the denominator; response adjustment is a separate assumption. Nonresponse analysis asks whether the respondents resemble the selected nonrespondents.

Report both group and total

Show each stratum’s population count, sample count, observed defect count and uncertainty, then the weighted total. A small stratum can have a noisy rate even when its overall weighted contribution is modest. If sampling occurred in clusters, such as stores or agents, a simple independent-case standard error is too optimistic. Cluster-aware uncertainty should match the actual draw.

Implementation

def stratified_defect_rate(strata):
    population = sum(group["population_count"] for group in strata)
    if population == 0 or any(group["sample_count"] == 0 for group in strata):
        raise ValueError("empty population or stratum sample")
    return sum(group["population_count"] * group["defect_count"] / group["sample_count"]
               for group in strata) / population

assert stratified_defect_rate([
    {"population_count": 1200, "sample_count": 120, "defect_count": 24},
    {"population_count": 300, "sample_count": 60, "defect_count": 24},
]) == 0.24

Performance and operating cost

The estimate scans S strata in O(S) time and O(1) auxiliary space. Drawing samples and computing design-based uncertainty cost more; large weights reduce effective information and should be reported.

Common Mistakes

Do not pool oversampled strata without weights.
Do not assume stratification removes frame or nonresponse bias.
Do not use a case-level standard error for clustered selection.