EWMA monitoring for small persistent rate shifts

Last updated: 5 Oct 20265 min read

tutorial

An exponentially weighted moving average combines the current rate with prior monitored values to surface sustained changes that may not trigger a single-day rule.

Define the monitored series

Suppose each day contains exactly 200 eligible support cases and the established late-response share is 0.05. Let the daily observed share be x. An EWMA updates z to 0.25 times x plus 0.75 times the previous z, starting at the frozen baseline center. A single higher day moves z partway; several higher days accumulate. This is useful for persistent small shifts, provided the baseline process and daily subgroup definition are stable. The target remains a separate operational threshold.

Use limits for the statistic being charted

The EWMA has less variation than a raw daily proportion because it averages information over time. With independent equal-size daily proportions and baseline Bernoulli variance, its time-t standard deviation includes the factor lambda divided by (two minus lambda) and a startup term that approaches one. Compare z with limits built for z, not the wider p-chart limits for one raw day. A chart can signal from a sustained rise even when each daily value individually stays inside its own limits.

Preserve the time order

Do not sort days by rate or drop quiet days before updating the state. A late-arriving daily batch requires a controlled replay from the last trusted checkpoint, not insertion into a finished sequence without recalculation. The EWMA value depends on every earlier monitored point and on its initialization. Store baseline version, lambda, last processed date, subgroup size and the sequence of values used to reach the current state. A snapshot supports replay.

Do not apply fixed-size limits blindly

If daily eligible counts range from 40 to 700, the variance of each daily proportion changes. The fixed-n formula below is intentionally restricted to equal-size teaching subgroups. In production, use a monitored statistic and limits designed for variable precision, stabilize subgroup sizes where appropriate, or model daily counts directly. Seasonality, serial correlation and changes in queue mix can also invalidate simple independent-day limits.

Treat signals as tests of process behavior

The code feeds ten days near 0.04 followed by eight days at 0.09, with a baseline center of 0.05. It checks that the latter stretch generates at least one upper signal. That assertion is a code exercise, not evidence that a real service process changed. Investigate logging, workload, staffing and case mix before action. The alert workflow records that decision.

Implementation

from math import sqrt

def ewma_rate_signals(daily_rates, baseline_rate, daily_count,
                      weight=0.25, sigma_width=3):
    if not 0 < weight <= 1 or daily_count <= 0:
        raise ValueError("invalid EWMA configuration")
    if not 0 < baseline_rate < 1 or any(not 0 <= rate <= 1 for rate in daily_rates):
        raise ValueError("invalid rates")
    if sigma_width <= 0:
        raise ValueError("invalid limit width")
    daily_sigma = sqrt(baseline_rate * (1 - baseline_rate) / daily_count)
    smoothed = baseline_rate
    results = []
    for day_index, observed_rate in enumerate(daily_rates, start=1):
        smoothed = weight * observed_rate + (1 - weight) * smoothed
        ewma_sigma = daily_sigma * sqrt(
            weight / (2 - weight) * (1 - (1 - weight) ** (2 * day_index)))
        lower = max(0.0, baseline_rate - sigma_width * ewma_sigma)
        upper = min(1.0, baseline_rate + sigma_width * ewma_sigma)
        results.append((day_index, smoothed, lower, upper,
                        smoothed < lower or smoothed > upper))
    return results

series = [0.04] * 10 + [0.09] * 8
signals = ewma_rate_signals(series, 0.05, 200)
assert any(day > 10 and flagged for day, _, _, _, flagged in signals)

Performance and operating cost

Updating D ordered days is O(D) time and O(D) space if every state is retained; keeping only the latest state is O(1) space. Replaying after a backfill is O(D) from the checkpoint. The equal-size independent-day variance assumption is the main limit, not computation.

Common Mistakes

Do not compare an EWMA state with raw daily p-chart limits.
Do not reorder days or ignore late backfills.
Do not use the fixed-count formula when subgroup sizes change materially.