Retention cohorts: return behavior only after a full observation window

Last updated: 7 Oct 20265 min read

tutorial

A retention rate compares members who entered together and had the same opportunity to return.

Anchor membership once

For an acquisition cohort, assign each eligible user to the week of first verified purchase. Do not move the user to a new cohort after a later purchase or identity merge without versioning the report. Retention needs a declared return event: another paid order, any app visit or an active subscription are different outcomes. The event contract must distinguish them.

Specify the return interval

Week-one retention can mean activity in days 7 through 13 after entry, activity by day 14, or activity during the next calendar week. Choose one. For the first definition, a user with a return on day 4 does not qualify unless they return again in days 7 through 13. Use the same timezone and day-boundary convention for all users. A calendar-week chart can otherwise mix seven-day and one-day opportunities.

Exclude immature cohorts from the comparable rate

A user acquired nine days before cutoff has not completed the full days-7-through-13 window. Include that user in the cohort count but mark their retention outcome pending. Calculate the observed week-one rate only over users with enough follow-up, and show pending count. Dropping the pending row from the cohort entirely hides acquisition volume. Cohort entry and censoring explain why this distinction matters.

Keep composition beside retention

Compare acquisition channel, first-order value band and region across cohorts. A recent campaign may bring more one-time buyers and lower the pooled return rate while every channel’s within-channel rate remains stable. Standardize only when the target question calls for a fixed mix; retain the actual cohort rate for operational planning. Do not claim a product change caused the shift from a before-and-after table.

Make a hand-checkable example

In a cohort of 47 acquired users, 38 have completed the full week-one window. Eleven of those 38 made a qualifying return; nine remain pending. The observed mature-user return fraction is 11/38. Counting pending users as nonreturning gives 11/47 and answers a different, premature question. Publish both the mature denominator and pending count.

Implementation

def week_one_retention(purchases, acquisition_day, analysis_day):
    matured = acquisition_day + 14 <= analysis_day
    if not matured:
        return None
    return any(7 <= purchase_day - acquisition_day < 14
               for purchase_day in purchases)

assert week_one_retention([3, 9], 0, 20) is True
assert week_one_retention([3], 0, 20) is False
assert week_one_retention([9], 0, 10) is None

Performance and operating cost

Checking P purchase days for one user costs O(P) time and O(1) auxiliary space. A warehouse implementation groups purchases by stable user ID and cohort date; sorting or indexing makes interval lookups faster, but identity merges and late events can still restate historical cohorts.

Common Mistakes

Do not compare immature recent cohorts with fully observed older cohorts.
Do not call any return within 14 days week-one retention without defining the interval.
Do not assign a repeat buyer to a new acquisition cohort.