Survey inclusion probabilities and base weights

Last updated: 7 Oct 20265 min read

tutorial

A base weight is the inverse of a known selection probability, connecting sampled units to the population they represent.

Specify the frame and unit

The support-handoff survey frame contains 1,200 eligible cases: 800 standard and 400 partner. The design randomly invites 80 cases from each queue without replacement. The selection probability is 80/800, or 0.10, for a standard case and 80/400, or 0.20, for a partner case. One case is the sampling unit, even if a customer opened multiple cases; customer-level claims would require a different design. Frame coverage limits what the weights can represent.

Compute inverse selection weights

A selected standard case represents 1/0.10, or 10, frame cases. A selected partner case represents 1/0.20, or five. Multiplying each 80-case invitation count by its base weight reconstructs 800 and 400, for a total of 1,200. This is a design check, not evidence that respondents mirror the frame. If cases were sampled with replacement or through multiple routes, their inclusion probabilities need the actual design formula.

Distinguish invitation and respondent sets

The base weight belongs to an invited unit before response is known. Suppose 48 standard and 32 partner invitees answer. Applying base weights only to those 80 respondents yields represented totals of 480 and 160, which do not sum to the 1,200-case frame. Response adjustment is a separate step with its own assumptions. Never call the base weight a cure for nonresponse.

Keep design metadata at record level

Store stratum, frame size, sample size, selection probability, base weight and draw identifier beside every invitation. A report should preserve the date and eligibility version of the frame. If a case became ineligible after drawing, document whether it was an out-of-scope unit or a nonrespondent; those changes alter denominators and adjustment logic. Stratified sampling supplies the broader design context.

State the estimand

A weighted case-level share estimates a proportion among frame cases under the probability design and valid response assumptions. It does not automatically estimate a proportion among unique customers or all support interactions. If one queue has a different outcome definition, reweighting cannot make the measures comparable. The survey contract determines what the item actually measures.

Implementation

def invitation_design(frame_counts, invitation_counts):
    plan = {}
    for queue, frame_size in frame_counts.items():
        invited = invitation_counts.get(queue, 0)
        if frame_size <= 0 or not 0 < invited <= frame_size:
            raise ValueError("invalid frame or invitation count")
        probability = invited / frame_size
        plan[queue] = {"inclusion_probability": probability,
                       "base_weight": 1 / probability,
                       "reconstructed_frame": invited / probability}
    if set(invitation_counts) - set(frame_counts):
        raise ValueError("invitation queue missing from frame")
    return plan

design = invitation_design({"standard": 800, "partner": 400},
                           {"standard": 80, "partner": 80})
assert design["standard"]["base_weight"] == 10
assert design["partner"]["base_weight"] == 5
assert sum(item["reconstructed_frame"] for item in design.values()) == 1200

Performance and operating cost

For G sampling strata, computing design weights is O(G) time and output space. Joining weights to I invitation records is O(I) expected time with keyed lookup. Correct variance estimation must also retain the actual sampling design, not only final weights.

Common Mistakes

Do not infer a respondent weight directly from selection probability alone.
Do not use a case-level base weight for a unique-customer estimand.
Do not discard frame and draw metadata after calculating one weighted percentage.