Project: evaluate a lesson-prerequisite graph recommender

Create the graph packet

Define lesson, concept and learner nodes with typed directed edges for teaches, requires and completed. Give every edge known-at time and event identity. Include one duplicate, one newly published lesson, one withdrawn lesson and a learner completion after the historical decision cutoff. The snapshot contract decides what a model can observe.

Establish baselines

First rank eligible lessons by shared prerequisite concepts and a simple editorial route. Then implement a one-hop or two-hop neighborhood feature; a graph neural network is optional only if the simpler baseline leaves a measured gap. Deduplicate neighbors and cap high-degree expansion. Message passing must not include the target completion edge.

Evaluate honestly

Use later completion edges as positives and build eligible negative candidates under the past catalog snapshot. Report candidate recall, ranking recall at five and coverage by new versus established lessons. Do not call every unexposed pair a negative. Keep the newest period untouched while choosing graph features or fanout.

Submit release evidence

Deliver schema, graph versions, split IDs, code, model or rule manifest, subgroup scores and a serving trace. Assert that the withdrawn lesson is gated out, the new lesson has a fallback, and version swap or rollback preserves coherent responses. A reviewer should reconstruct one recommendation from the logged graph and candidate IDs.

Implementation

def prerequisite_overlap(required_concepts, learned_concepts):
    required = set(required_concepts)
    learned = set(learned_concepts)
    if not required:
        return 1.0
    return len(required & learned) / len(required)

assert prerequisite_overlap({"data-grain", "time-split"}, {"data-grain"}) == 0.5

Performance and operating cost

Set overlap costs O(R + L) time and space for R required and L learned concepts. Full graph serving adds neighborhood and ranking cost; measure its gain over this baseline before accepting a larger graph index and retraining process.

Common Mistakes

Do not let a future completion enter a past graph snapshot.
Do not treat an unpublished lesson as a negative candidate.
Do not deploy without a withdrawal gate and cold-node fallback.