Project: calibrate an autoencoder alert for freezer telemetry

Partition the operating record

Collect temperature, compressor current, door state and three other measurements in fixed windows. Keep maintenance windows and confirmed faults out of the normal fitting pool, then split by device and day before scaling. Reserve separate normal calibration days, incident evaluation days and a final untouched test group. Multiple overlapping windows from one day must not appear on both sides of a split. Masked scoring needs an observation flag for every feature.

Fit and record a baseline

Compute feature medians and scales from training normals only. Fit a six-to-two-to-six autoencoder and compare it with a simple standardized-distance rule. Loss excludes missing targets and rejects an all-missing window. Keep the reconstruction loss, feature coverage and score distribution by device; a low training loss alone does not establish fault separation. Run a controlled VAE candidate with named reconstruction and KL terms, but keep its serving score policy fixed. KL accounting prevents a hidden change in scale.

Calibrate the alert budget

Choose a score threshold on normal calibration device-days for a stated false-alert budget, such as no more than three alerts per device-day under the observed sample. Group consecutive flagged windows into one incident before counting alerts. Evaluate fault recall and time to first alert on a separate incident set. A toy four-window quantile in code only illustrates an API; real tail calibration needs far more independent days and a confidence statement.

Attack the decision rule

Withhold the compressor-current reading, then repeat with all fields missing; the first should be scored with coverage attached, and the second rejected. Test a door-open service period, a sensor replacement and a genuine warming fault. An overly capable decoder may reconstruct a warming fault, while a new sensor offset may create false positives. Report those failures explicitly instead of tuning the threshold on the final test days.

Package a reversible release

Save weights, feature order, medians, scales, missing-value policy, threshold, aggregation window and model revision. On a fresh process, reload and reproduce fixed-window scores and alert decisions. Monitor score and coverage distributions by device cohort and compare confirmed incidents against the expected alert budget. If a drift appears, route it to evaluation; do not silently raise the threshold. Keep the preceding artifact available for rollback.

Implementation

import torch
from torch import nn

class FreezerWindowAutoencoder(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder = nn.Sequential(nn.Linear(6, 3), nn.ReLU(), nn.Linear(3, 2))
        self.decoder = nn.Sequential(nn.Linear(2, 3), nn.ReLU(), nn.Linear(3, 6))

    def forward(self, normalized_window: torch.Tensor) -> torch.Tensor:
        return self.decoder(self.encoder(normalized_window))

torch.manual_seed(71)
training_normals = torch.rand(8, 6)
calibration_normals = torch.rand(5, 6)
candidate_windows = torch.rand(2, 6)
detector = FreezerWindowAutoencoder()
optimizer = torch.optim.AdamW(detector.parameters(), lr=0.0037)
for _ in range(23):
    optimizer.zero_grad(set_to_none=True)
    reconstruction = detector(training_normals)
    loss = (reconstruction - training_normals).square().mean()
    loss.backward()
    optimizer.step()

detector.eval()
with torch.inference_mode():
    calibration_scores = (detector(calibration_normals) - calibration_normals).square().mean(dim=1)
    candidate_scores = (detector(candidate_windows) - candidate_windows).square().mean(dim=1)
threshold = torch.quantile(calibration_scores, 0.8)  # API illustration only
candidate_alerts = candidate_scores > threshold
assert candidate_alerts.shape == (2,)
assert torch.isfinite(candidate_scores).all()

Performance and operating cost

Dense model work is small for six features; data preparation, storage and operational alert review can dominate system cost. Online scoring is O(DH + HZ + ZH + HD) for feature width D and internal widths H and Z. Calibration and evaluation cost grows with independent device-days, not just window count. This toy quantile is deliberately too small for deployment; estimate threshold uncertainty and alert clustering on real historical days.

Common Mistakes

Do not calibrate and evaluate on overlapping windows from the same device-day.
Do not turn an all-missing window into a low-error normal score.
Do not equate one flagged window with one operational incident when alerts cluster.