Project: review a prior-sensitive helpdesk escalation decision

Last updated: 7 Oct 20265 min read

project

Compare prior assumptions, check queue-level replication and delay a costly intervention when the model frame is inadequate.

Write the decision before updating

The helpdesk considers adding a night-shift reviewer if one-day escalation risk is high enough. Define eligible tickets, mature one-day outcomes, the labor cost and the loss from unreviewed escalations. Use the same cost rule for every candidate prior. Nine escalations among 47 mature tickets are available for the first analysis; repeat contacts are grouped by customer for a later dependence review. Prior sensitivity begins with an explicit operational threshold, not a retrospective prior choice.

Compare plausible prior positions

A low-concentration prior and a more skeptical prior yield posterior means on opposite sides of the cost threshold. Record both and mark the decision prior-sensitive. The point estimates differ because 47 tickets do not overwhelm every defensible prior concentration. Plot or tabulate prior predictive ticket counts and reject a prior only for a documented mismatch, not because it blocks the preferred action. Posterior decision summaries should use the same loss assumptions.

Check queue structure

Split the mature cases into day and night queues. The observed gap is large enough to warrant a posterior predictive group check under the pooled-rate model. Reproduce each queue count and outcome window in simulations; save the seed and a distribution of replicated gaps. If the model cannot represent the pattern, a pooled intervention rule may be wrong for both queues. Investigate queue-specific labels, agent staffing and customer mix. The predictive check is a model criticism tool, not proof of a night-shift cause.

Publish a bounded disposition

Hold automatic staffing changes while the prior-sensitive decision and queue discrepancy are reviewed. Continue collecting mature outcomes under the frozen definition, consider a stratified model if sample support permits, and set a revisit date. Include prior shapes, group denominators, posterior means, decision costs, predictive-check result and data owner. The result is still useful: it identifies exactly which assumption controls the proposed action. The target frame must remain fixed across the next sample.

Implementation

def reviewer_decision(prior_options, escalations, tickets, no_review_cost,
                      review_fixed_cost, reviewed_case_cost):
    choices = []
    for alpha, beta in prior_options:
        risk = (alpha + escalations) / (alpha + beta + tickets)
        no_review = no_review_cost * risk
        add_reviewer = review_fixed_cost + reviewed_case_cost * risk
        choices.append("add-reviewer" if add_reviewer < no_review else "keep-current")
    return choices[0] if len(set(choices)) == 1 else "review:prior-sensitive"

decision = reviewer_decision([(2, 18), (4, 56)], 9, 47, 8, 0.7, 3)
assert decision == "review:prior-sensitive"

Performance and operating cost

Computing decisions for k candidate priors costs O(k) time and O(k) space. The greater cost is validating ticket maturity, queue heterogeneity and staffing impact. An automatic change based on a single pooled posterior can spend labor on the wrong shift or miss the queue with the highest actual burden.

Common Mistakes

Tuning the prior after seeing which staffing decision it produces.
Using tickets without mature escalation outcomes in the denominator.
Assuming a pooled rate describes both queues despite a failed group check.
Equating a prior-sensitive decision with proof that data are useless.