Aspect sentiment: bind opinions to targets and negation

Last updated: 6 Oct 20265 min read

tutorial

A customer can praise delivery and criticize billing in one message. A single positive label loses the action the product team needs.

Define the annotation unit

Store a tuple of document revision, aspect target span, opinion span, polarity and reviewer status. An aspect may be explicit (“billing”) or implied by a phrase such as “charged twice”; document which kinds are allowed. A message-level label is a separate annotation. Do not let an inferred overall mood overwrite a concrete complaint. The offsets must select the captured original text, following the span contract, even when a normalized view is used for features.

Scope negation and contrast

“The refund was not slow” is not a negative opinion about refunds; “the refund was slow, but the agent was helpful” contains two targets and different orientations. Annotate the cue, its scope and the target it modifies. A keyword list that counts slow without its negator will invert the first judgment. Sarcasm and quotation require an uncertainty label or review path, not an invented confidence boost. Preserve punctuation and clause boundaries through text intake.

Make the label policy operable

Define positive, negative, neutral, mixed and not-applicable with examples from the product domain. Neutral means an opinion is present without clear polarity; not-applicable means no opinion about that aspect was expressed. Reviewers should mark a cancellation request as an intent, not automatically negative sentiment. Separate customer-authored words from copied agent replies and templates. Group repeated conversations during evaluation so a boilerplate sentence cannot appear on both sides of a split.

Evaluate the relation, not just the words

A prediction is useful only if it attaches the right polarity to the right aspect. Report target detection recall, exact target span F1 and polarity accuracy conditional on a correctly matched target, plus end-to-end tuple F1. Review failures on negation, comparative statements, code-switching and multi-aspect messages. Follow calibration and slice evaluation before using a score to prioritize a queue. The feedback project turns the annotation contract into a release test.

Implementation

def validate_opinion(message, opinion):
    target_start, target_end = opinion["target_span"]
    cue_start, cue_end = opinion["opinion_span"]
    for start, end in ((target_start, target_end), (cue_start, cue_end)):
        if not 0 <= start < end <= len(message):
            raise ValueError("opinion offset outside original message")
    if opinion["polarity"] not in {"positive", "negative", "neutral", "mixed"}:
        raise ValueError("unknown polarity")
    return message[target_start:target_end], message[cue_start:cue_end]

review = "Billing is slow, support is helpful."
assert validate_opinion(review, {"target_span": (0, 7), "opinion_span": (11, 15),
                                 "polarity": "negative"}) == ("Billing", "slow")

Performance and operating cost

Span validation is O(a) operations for a annotated aspect records plus the characters copied for selected spans. A pair model may score each aspect against each candidate opinion; unconstrained pairing is O(a·o) for a targets and o opinion phrases. Cap pairs with reviewed boundaries and measure candidate recall. The expensive operating cost is disagreement review, so track how often polarity or target scope needs adjudication.

Common Mistakes

Using one overall sentiment label when the message has conflicting aspects.
Counting a negated keyword as an ordinary negative cue.
Treating absence of an opinion as neutral sentiment.
Evaluating polarity without checking whether it was attached to the correct target.