Graph-grounded retrieval: candidate recall and unsupported claims

Last updated: 5 Oct 20265 min read

tutorial

A graph can guide retrieval, but a generated answer still needs evidence for every factual relationship it states.

Separate graph from text

The graph is a curated index of entity relationships, while lesson passages carry explanations and qualifications. Use a recognized entity to retrieve graph neighbors, then fetch the exact supporting lesson revisions. An extracted edge without verified evidence should not be presented as a fact merely because it appears in a path. Retrieval evaluation should check candidate supply before judging answer wording.

Measure candidate coverage

If the correct lesson or evidence claim is absent from the graph, a graph-only retriever cannot recover it. Track entity-linking accuracy, graph candidate recall and passage recall separately. Compare with a text-only baseline; graph expansion can improve disambiguation but may also introduce irrelevant neighbors. Keep a fallback when a new lesson has no graph edges.

Audit answer support

For each answer statement, record the claim ID, evidence revision and retrieved passage that supports it. If the passage contradicts an older edge, abstain or route to review instead of inventing a reconciliation. Claim timing matters when a current answer and a historical answer refer to different curriculum versions.

Test a conflict

Create a graph edge saying lesson A requires concept C, then remove that prerequisite in the latest lesson revision while leaving an old evidence span indexed. A current answer should not assert the requirement. The evaluation should identify stale evidence, not treat the existence of a graph path as a successful grounded answer.

Implementation

def supported_claim_ids(answer_claim_ids, evidence_by_claim):
    return [claim_id for claim_id in answer_claim_ids
            if claim_id in evidence_by_claim
            and evidence_by_claim[claim_id]["reviewed"]
            and evidence_by_claim[claim_id]["current"]]

Performance and operating cost

Checking A proposed claims is O(A) expected time with an evidence index and O(S) output space for S supported claims. Full retrieval adds graph traversal, passage search and version checks; measure each stage independently.

Common Mistakes

Do not call a graph path sufficient evidence for a generated statement.
Do not hide a missing graph candidate behind fluent answer text.
Do not use stale passage revisions for current claims.