Posterior intervals and threshold decisions

Report an interval with its model

After six escalations among 47 audited cases and a Beta(3, 27) prior, the posterior is Beta(9, 68). An equal-tail 80% credible interval encloses the middle 80% of probability under that posterior. Its interpretation is conditional on the prior, the binomial likelihood, valid labels and the selected cohort. It is not a promise that 80% of future observed audit shares fall in that interval.

Compute a probability that matches the action

Suppose leadership adds review capacity if the escalation rate probably exceeds 15%. Calculate posterior probability that the underlying rate exceeds 15%, then compare it with a decision threshold fixed before seeing the result. That probability is about the model parameter, not the probability that a randomly selected case escalates. The action objective should explain why a threshold crossing changes staffing.

Separate parameter and predictive uncertainty

A future 47-case audit has two sources of variation: uncertainty about the rate and random case outcomes conditional on that rate. A credible interval for the rate does not directly describe future escalation counts. To forecast counts, draw a rate from the posterior and then draw binary outcomes, or use the beta-binomial predictive distribution. Check predicted counts against held-out periods where possible.

Treat Monte Carlo as an approximation

A standard-library sampler can draw many rates from the beta posterior. Sort them for equal-tail quantiles and count draws above a threshold. Repeating with a fixed seed makes a lesson reproducible, while a larger number of draws reduces Monte Carlo noise. A production report should state the random seed, draw count and numerical precision, or use a vetted distribution implementation.

Test prior sensitivity

Recalculate interval and threshold probability under a weaker reasonable prior. If the staffing decision changes, the sample has not overwhelmed prior choice; say so. The prior predictive check helps rule out implausible priors before the audit. Never substitute a posterior probability for the cost analysis of false alarms and missed escalations.

Implementation

from random import Random

def posterior_rate_summary(alpha, beta, threshold, draws=20000, seed=47):
    if alpha <= 0 or beta <= 0 or not 0 <= threshold <= 1 or draws < 100:
        raise ValueError("invalid posterior summary request")
    generator = Random(seed)
    rates = sorted(generator.betavariate(alpha, beta) for _ in range(draws))
    return {"mean": alpha / (alpha + beta),
            "middle_80": (rates[int(draws * 0.10)], rates[int(draws * 0.90)]),
            "probability_above": sum(rate > threshold for rate in rates) / draws}

summary = posterior_rate_summary(9, 68, 0.15)
assert 0 < summary["middle_80"][0] < summary["mean"]
assert summary["mean"] < summary["middle_80"][1] < 1
assert 0 <= summary["probability_above"] <= 1

Performance and operating cost

Generating and sorting S beta draws costs O(S log S) time and O(S) memory. A direct distribution quantile and survival function can avoid simulation for this conjugate case. The example intentionally keeps the numerical method visible.

Common Mistakes

• Do not describe a credible interval as a confidence interval with identical interpretation.
• Do not read a parameter probability as a future case probability.
• Do not choose a decision cutoff after inspecting which action the posterior favors.

Read next

• Prior predictive checks for a binary service rate
• Beta-binomial updating with an auditable case count
• Shared priors, shrinkage and the limits of fixed pooling
• Project: Bayesian monitoring of support escalations

Continue the workflow: Held-out predictive log scores for a binary rate model.