Domain word senses: inventory, abstention and new meanings

Last updated: 6 Oct 20265 min read

tutorial

A term can denote different operational concepts inside one company. Name the senses before training a classifier or joining search results.

Define senses at the decision boundary

In support logs, “charge” may mean a card debit, stored battery energy or an instruction to bill. A dictionary entry is too broad for deciding whether a billing runbook should appear. Assign stable sense IDs with short definitions, inclusion rules, exclusions and two independently written examples. Include the downstream action each sense permits. The lemma and part-of-speech contract identifies a word form; it does not decide which meaning a particular use carries.

Keep an explicit unknown state

Do not force every occurrence into the current inventory. A new product can introduce a meaning that was absent when labels were collected; the phrase “reserve charge” could name a feature rather than a payment. Return NIL with the observed span, context, candidate scores and inventory version. Route repeated NIL cases to a reviewer, who can create a new sense or correct a bad candidate. The entity-link NIL decision is related, but identifies a referent rather than a lexical meaning.

Version the inventory like a schema

Splitting a broad PAYMENT sense into CARD_DEBIT and ACCOUNT_CREDIT changes old training labels and search filters. Store an immutable inventory version on annotations, indexes and predictions. Define a mapping for migration, and leave ambiguous old rows unresolved when evidence is insufficient. Evaluate the new inventory on a frozen sample before reindexing. Rewriting historical labels in place makes earlier quality reports irreproducible and may silently change what a saved search returns.

Audit impact, not just agreement

Sample high-frequency terms, rare meanings, negated spans and language variants. Count NIL rate, wrong-sense retrieval joins and reviewer disagreements by business workflow. A sense model with high overall accuracy can still surface a billing article for a battery incident if the common sense dominates evaluation. Hybrid retrieval should retain sense evidence as one feature while preserving document access rules. An uncertain meaning should not bypass those rules.

Implementation

SENSES = {
    "charge": {
        "payment-debit": {"card", "invoice", "refund"},
        "battery-energy": {"battery", "voltage", "charger"},
    }
}

def sense_candidates(term, context_tokens):
    context = set(context_tokens)
    scores = [(sense_id, len(context & clues))
              for sense_id, clues in SENSES.get(term, {}).items()]
    if not scores:
        return {"state": "nil", "candidates": []}
    best = max(score for _, score in scores)
    winners = [sense_id for sense_id, score in scores if score == best]
    if best == 0 or len(winners) != 1:
        return {"state": "review", "candidates": winners}
    return {"state": "proposed", "sense_id": winners[0]}

assert sense_candidates("charge", ["card", "refund"])["sense_id"] == "payment-debit"
assert sense_candidates("charge", ["card", "battery"])["state"] == "review"

Performance and operating cost

The small illustrative scorer compares s senses against c context tokens, taking O(s × c) time in the worst case and O(c + s) temporary space. A production system may use indexed features or a learned ranker. This code is a review-oriented baseline, not a calibrated probability; measure the cost of wrong retrieval separately from the cost of abstention.

Common Mistakes

Treating a lemma as a complete sense label.
Silently mapping unseen meanings to the most frequent sense.
Changing sense definitions without versioning old annotations.
Measuring only overall accuracy when a rare meaning triggers costly actions.