Project: build a guarded next-lesson recommender

Prepare the manifest

Create a catalog with paths, subject, prerequisites, publication state and effective time. Add learner progress and an impression log with rank, click, completion and policy version. Include a new lesson with no interactions, a learner with no history and a withdrawn page. The outcome contract must name the metric window and denominator.

Assemble the feed

Nominate candidates from current-page concepts, prerequisites and an interaction-based route when history supports it. Deduplicate, apply hard eligibility and rank a short slate. Cap repeated subjects, then validate publication state again before serving. Keep an explicit empty state if no candidate survives. Slate constraints must be visible in the submitted trace.

Evaluate without hindsight

Use a chronological split and reconstruct learner history and catalog state at each decision. Report candidate recall, recall at five, catalog coverage and outcome rates for sparse learners separately. Do not turn undelivered candidates into negatives. Compare with a simple editorial-prerequisite baseline, and describe what offline logs cannot establish about a new policy’s causal effect.

Submit checks

Deliver code, synthetic fixture, state manifest, example slate traces and a short release decision. Assert that the withdrawn page never appears, the cold-start learner receives a valid fallback and duplicate nominations occupy one slot. Include a privacy/deletion case and a recorded rollback path. Canary and rollback should gate any real serving change.

Implementation

def safe_lesson_slate(nominations, published_paths, completed_paths, limit=5):
    if limit < 1:
        raise ValueError("limit must be positive")
    selected = []
    seen = set()
    for lesson_path in nominations:
        if lesson_path in seen or lesson_path not in published_paths or lesson_path in completed_paths:
            continue
        seen.add(lesson_path)
        selected.append(lesson_path)
        if len(selected) >= limit:
            break
    return selected

assert safe_lesson_slate(["ai/first", "ai/first", "ai/retired"], {"ai/first"}, set()) == ["ai/first"]

Performance and operating cost

A simple nomination filter is O(M) expected time with O(K) retained state; ranking K items is usually O(K log K). The project’s real cost lies in time-correct logging and catalog snapshots, which must be budgeted before a learned model is added.

Common Mistakes

Do not let an offline score override publication state.
Do not infer dislike from lack of exposure.
Do not ship without a cold-start and rollback path.