Survey constructs, response units and recall windows

Last updated: 5 Oct 20265 min read

tutorial

A survey item is a measurement rule: define the construct, who can answer it and the period the answer covers.

Define the construct before drafting

A support team wants to know whether a customer handoff was clear. “How happy were you with support?” measures a broader impression; it cannot isolate handoff clarity. A narrower item asks about the most recent handoff between support teams. Its target is the respondent’s reported understanding of the next step after that handoff. This remains self-report, not a direct measure of actual resolution quality. Proxy measurement explains why the distinction matters.

Set a response unit

Choose one eligible support case and one customer contact as the response unit. If a customer has three cases, the invitation must name the selected case; otherwise different respondents may describe different experiences. The analysis table needs a case key, invitation key, survey version and answer timestamp. Do not let repeated clicks create duplicate respondents. Dataset grain keeps invitation and response counts separate.

Make the recall period answerable

Ask about a handoff in the last 14 days and show the case date. A respondent who cannot recall the event needs an explicit option, not a forced guess. Short recall windows can improve specificity but may reduce eligible sample size; long windows increase coverage while making memory errors more likely. Document the tradeoff and keep the same window when comparing periods. An answer collected after a policy change may describe an older experience.

Remove implied judgment

A question such as “Why did our helpful agent make the handoff easy?” presumes both helpfulness and ease. Ask one idea at a time: “After the handoff, how clear was the next step?” Then separately ask whether the next step actually occurred. Avoid naming a desired answer in the invitation. Test the item with people who experienced a handoff, including those whose case was unresolved.

Write a measurement contract

Store item text, displayed context, eligibility, language, recall window, permitted answers and analysis interpretation under a version ID. If the item changes, retain both versions in raw data. Response options may change the meaning as much as the question wording. A report should say “reported handoff clarity among respondents,” not “handoff quality among all customers,” unless coverage and response assumptions support that stronger claim.

Implementation

from datetime import date

def eligible_handoff_case(handoff_date, invitation_date, days=14):
    if days <= 0:
        raise ValueError("recall window must be positive")
    elapsed = (invitation_date - handoff_date).days
    return 0 <= elapsed <= days

invitation = date(2026, 9, 27)
assert eligible_handoff_case(date(2026, 9, 18), invitation)
assert not eligible_handoff_case(date(2026, 9, 2), invitation)
assert not eligible_handoff_case(date(2026, 9, 28), invitation)

Performance and operating cost

Eligibility checking is O(1) per case and O(N) for N invitations. Joining responses to cases may dominate runtime and must preserve the one-case response unit. The main cost is question testing and maintaining versioned measurement definitions.

Common Mistakes

Do not report self-reported clarity as verified resolution quality.
Do not let each respondent choose an unspecified handoff event.
Do not change recall windows between reporting periods without flagging the break.