Project: route multi-issue tickets through a label hierarchy

Last updated: 7 Oct 20265 min read

project

Propose all supported support queues, retain parent-only outcomes and stage taxonomy changes before agents receive new routes.

Define the decision

A ticket can request a refund while reporting an account lockout. The service should propose both branches when evidence supports them, while leaving uncertain specialist leaves at the parent or sending the case for review. It cannot execute account or payment changes. Return label IDs, ancestor paths, score bundle, taxonomy revision and reason for each chosen or withheld leaf.

Build a reviewed corpus

Sample single-issue, multi-issue and ambiguous cases, including repeated conversations and rare high-cost queues. Reviewers annotate independent branches and parent-only labels under one frozen taxonomy. Group customer events before splitting. The label contract makes child-parent consistency explicit, and migration review handles taxonomy updates.

Compare operating costs

Start with a flat baseline and compare a hierarchy-aware model using the same audit. Score branch precision, leaf recall, wrong specialist handoff, parent-only review rate and agent time. Report per-language performance. A model that returns every plausible queue may have high recall and unusable workload; cap routes under a written policy and show uncertainty instead of flooding teams.

Release and correct

Shadow-route first. Version model, taxonomy and thresholds together, then review conflicts and high-confidence wrong routes. Preserve historical labels when queues merge or split. A reviewer correction should update the ticket decision and a separate learning record; do not silently rewrite the original prediction. On drift, revert the affected bundle and route uncertain cases to agents.

Implementation

def admit_routing_labels(labels, parent_by_label, active_labels):
    unknown = set(labels) - active_labels
    if unknown:
        return {"state": "review", "reason": "retired-label"}
    complete = set(labels)
    frontier = list(labels)
    while frontier:
        parent = parent_by_label.get(frontier.pop())
        if parent and parent not in complete:
            complete.add(parent)
            frontier.append(parent)
    return {"state": "proposed", "labels": sorted(complete)}

parents = {"refund": "billing", "billing": "support",
           "access": "account", "account": "support"}
result = admit_routing_labels({"refund", "access"}, parents,
                              set(parents) | {"support"})
assert result["state"] == "proposed" and "billing" in result["labels"]

Performance and operating cost

The graph walk is O(v + e) over reached labels and parent links, with O(v) space for the result. Model inference and cross-team review dominate cost. Measure how often extra routes actually help resolve the ticket; adding labels without improving resolution can make agents do more work.

Common Mistakes

Dropping the second issue because the classifier expects one label.
Sending an uncertain leaf directly to a specialist queue.
Losing taxonomy revision from stored predictions.
Training on corrected labels without preserving the original prediction.