Relation extraction with direction, negation and evidence

Last updated: 7 Oct 20265 min read

tutorial

Two entities appearing in one sentence do not prove a relationship. Store direction, time, negation and the supporting span with each extracted edge.

Define a typed relation contract

An incident note might state “Service Delta depends on Queue K” or “Queue K no longer feeds Service Delta.” The entity pair alone is insufficient. Define allowed subject and object types, relation direction, effective time, negation and confidence. A relation is an assertion about a source revision, not a permanent graph fact. Reject arguments outside the allowed type set. Exact entity spans must be checked before relation scoring.

Construct examples without leakage

Annotate the minimal evidence span that supports the relation and mark explicit no-relation pairs. Include near misses: co-occurring names, quoted historical architecture, hypothetical changes and negated dependencies. Review entity coreference first when the relation uses “it” or “the processor.” Split by incident, document and time so the same configuration sentence cannot appear in both training and audit. A model that memorizes service pairs will otherwise look accurate without reading the relation words.

Evaluate both stages

Report entity recall, pair candidate recall, relation precision and end-to-end edge F1. An edge is correct only when subject, predicate, object, direction and evidence revision match the reviewed record. Partial entity overlap is not enough if it changes which system owns an action. Track errors caused by coreference separately from relation classification. The coreference path defines when a mention can be promoted to a catalog identity.

Publish as a reversible assertion

Write edges to a staging ledger with source digest and review state. If the source retracts a claim, mark the edge superseded instead of silently deleting audit history. Require human approval for high-impact dependencies or ownership claims. A graph query should distinguish “observed in source,” “reviewed current” and “retired.” The applied ledger makes those states visible; knowledge graph foundations cover later graph use.

Implementation

def relation_assertion(subject, predicate, object_id, evidence, allowed_types):
    subject_id, subject_type = subject
    target_id, target_type = object_id
    if (subject_type, predicate, target_type) not in allowed_types:
        raise ValueError("relation type or direction is not allowed")
    if not evidence["source_revision"] or not evidence["quote"].strip():
        raise ValueError("source evidence is required")
    return {"subject": subject_id, "predicate": predicate,
            "object": target_id, "evidence": evidence, "state": "staged"}

allowed = {("SERVICE", "DEPENDS_ON", "QUEUE")}
edge = relation_assertion(("service-delta", "SERVICE"), "DEPENDS_ON",
    ("queue-k", "QUEUE"), {"source_revision": "note-v7", "quote": "Delta depends on K"}, allowed)
assert edge["state"] == "staged"

Performance and operating cost

Contract validation is constant time per proposed edge with a hashed type set, aside from copying evidence text. Generating all entity pairs is O(e²) for e entities in a document; sentence windows and type constraints reduce candidates but can miss cross-sentence relations. Measure candidate recall before celebrating a faster classifier. Store source digests and staged edges to support revocation without losing provenance.

Common Mistakes

Inferring a relation from co-occurrence alone.
Dropping negation or swapping subject and object.
Scoring relation labels only on gold entity pairs.
Publishing a graph edge without source revision and retirement state.