Confounders and covariate timing: adjust only for variables with a valid role

Last updated: 7 Oct 20265 min read

tutorial

A covariate can help, harm or change a causal analysis depending on when it was measured and what caused it.

Draw the assumed paths

Prior learner activity can influence whether a person encounters the new panel and whether they complete lessons. It is a plausible pre-treatment confounder in observational data. By contrast, clicking the new panel occurs after treatment and may mediate its effect. Adjusting for that click can remove part of the effect being estimated. Record causal assumptions before selecting features.

Check timestamps

A “last seven days of activity” feature measured after panel launch contains treatment-period behavior. Freeze covariates at or before assignment and keep a source-availability timestamp. Causal feature timing shares the same future-information boundary as forecasting, though its purpose is identification rather than prediction.

Avoid collider selection

Suppose both motivation and panel quality influence whether a learner opens a follow-up survey. An analysis restricted to survey respondents can create a spurious relationship between motivation and treatment even if assignment was random. Prefer outcome collection for all assigned units, and report missingness by group rather than quietly filtering to complete cases.

Make the graph testable

List each variable, its measurement time, proposed role and whether it enters the adjustment set. Include prior activity, treatment assignment, panel click, completion and survey response. A reviewer should reject a post-treatment click used as a baseline covariate and ask how missing completions will be handled. Assignment checks do not fix outcome selection bias.

Implementation

def pre_assignment_covariates(covariates, assigned_at):
    accepted = {}
    for name, record in covariates.items():
        if record["measured_at"] >= assigned_at:
            raise ValueError(f"{name} was measured after assignment")
        accepted[name] = record["value"]
    return accepted

Performance and operating cost

Screening V covariates costs O(V) time and O(V) output space. The computational cost of a fitted adjustment model depends on its form, but no amount of fitting can identify an unmeasured confounder from timestamps alone.

Common Mistakes

Do not adjust for a mediator as though it were a baseline confounder.
Do not select only survey respondents without assessing selection.
Do not infer a causal graph from a correlation matrix.