Nonresponse adjustment cells and the positivity check

Last updated: 5 Oct 20265 min read

tutorial

A response adjustment increases respondent weights within a cell to account for invited units that did not answer, conditional on the cell definition.

Calculate each response factor

Of 80 standard invitees, 48 respond; of 80 partner invitees, 32 respond. Their cell response shares are 0.60 and 0.40. Multiply base weights 10 and five by inverse response shares: a standard respondent gets 10 × 80/48, about 16.67, while a partner respondent gets 5 × 80/32, or 12.5. The adjusted respondent weights sum to 800 standard and 400 partner frame cases. This arithmetic restores known queue totals, not unobserved opinions.

State the identifying assumption

Within each adjustment cell, response must be sufficiently unrelated to the survey outcome after conditioning on known variables for the adjustment to remove response bias. If customers with confusing handoffs are less likely to answer even within the standard queue, a queue-only adjustment still misses them. Compare respondents and nonrespondents on pre-invitation case features, then consider a better cell definition or follow-up sample. Nonresponse analysis addresses the residual risk.

Guard against empty cells

A cell with zero respondents has no observed survey outcome to upweight. Division by zero is a symptom of missing support, not a numerical nuisance to patch with an enormous weight. Combine cells only when the groups can reasonably share an outcome relationship, collect more responses, or report the cell as unresolved. Very small response counts produce unstable weights even when they are nonzero. Weight diagnostics quantify that instability.

Separate unit and item nonresponse

A completed survey may still skip the handoff-clarity item. The 48 and 32 counts address unit response; an item-level estimate may need another denominator and possibly another adjustment. Do not apply the same factor blindly to every question if eligibility and displayed branches differ. Response-state coding distinguishes skipped, not displayed and substantive answers.

Keep a release table

Publish frame counts, invitations, respondents, response shares, base weights and adjusted totals by cell. Verify that response IDs are unique and belong to sampled invitations. Record any cell collapsing and its rationale. If the fielding method changed halfway through, separate the periods before calculating a shared response factor. A weighted estimate is only as defensible as these provenance checks.

Implementation

def adjusted_queue_weights(frame_counts, invited_counts, response_counts):
    weights = {}
    for queue, frame_size in frame_counts.items():
        invited = invited_counts[queue]
        responded = response_counts[queue]
        if not 0 < responded <= invited <= frame_size:
            raise ValueError("empty or inconsistent response cell")
        base_weight = frame_size / invited
        weights[queue] = base_weight * invited / responded
    return weights

weights = adjusted_queue_weights({"standard": 800, "partner": 400},
                                 {"standard": 80, "partner": 80},
                                 {"standard": 48, "partner": 32})
assert round(weights["standard"], 2) == 16.67
assert weights["partner"] == 12.5
assert round(48 * weights["standard"] + 32 * weights["partner"]) == 1200

Performance and operating cost

For G adjustment cells, factor calculation is O(G) time and output space; attaching weights to R respondents is O(R) expected time. More detailed cells may reduce bias but increase variance and empty-cell risk. Variance estimation must respect both design and adjustment.

Common Mistakes

Do not treat adjusted queue totals as proof that opinions are unbiased within queues.
Do not generate a weight for a cell with no respondents.
Do not confuse survey completion with substantive response to every item.