Audio time-frequency masks and label preservation

Last updated: 6 Oct 20265 min read

tutorial

Spectral masking can regularize an audio model, but hiding the only labeled event turns a positive window into a contradictory training case.

Separate augmentation from missing data

A training-time frequency mask removes a band of spectral values; a time mask removes consecutive frames. These artificial omissions are not equivalent to microphone dropouts or packet loss in the field. Use them only after verifying the source label remains supported. A brief fault chirp might occupy four of eighty frames; masking those four frames erases the event. Keep reviewed event boundaries and cap mask size relative to the event interval. The window contract maps frames back to physical time.

Choose replacement values deliberately

Setting a log-power mask to numeric zero does not mean silence if the log scale was standardized. It may mean average energy. Choose a replacement consistent with the representation, and audit whether masked regions become an easy synthetic cue. Apply augmentation to training examples only; validation should reflect the intended field distribution unless a separate stress test is declared. Record seeds, mask widths and the proportion of examples whose label-bearing interval was touched. Never report augmented validation as if it were untouched deployment data.

Preserve timestamp alignment

If hop length is 160 samples at 16,000 samples per second, one frame step is ten milliseconds. A time mask from frames twenty through twenty-seven covers roughly a short region, but window support overlaps neighboring frames. Align event times to frame support rather than treating each frame as an isolated instant. The pure-Python code rejects a mask overlapping a reviewed event span before replacing values. A production implementation should also consider the window width and boundary rounding.

Evaluate rare-event recall

Compare clean validation and a controlled noise or dropout stress set by microphone type, fault length and background noise. Report event recall, false alarms per machine-hour and latency to first alert. A small aggregate accuracy gain can hide lower recall for short squeals. Track whether masking improves robustness only on synthetic corruption and degrades real recordings. The project includes a reviewed fault gallery and an alert-budget gate.

Keep a reproducible transform manifest

Package sample rate, spectral transform, normalization, mask distributions and augmentation order with the model. A waveform shift before spectral extraction and a time mask after extraction do not have identical effects. Run fixed seeds through the same input twice and assert reproducibility where required. Keep augmentation off at serving. If fault times are unavailable, use conservative masks and inspect positive crops manually rather than assuming their labels survive.

Implementation

def safe_time_mask(spectral_rows, start, stop, event_start, event_stop):
    if not (0 <= start <= stop <= len(spectral_rows[0])):
        raise ValueError("invalid frame interval")
    if max(start, event_start) < min(stop, event_stop):
        raise ValueError("mask would remove reviewed fault evidence")
    masked = [row.copy() for row in spectral_rows]
    for row in masked:
        row[start:stop] = [0.0] * (stop - start)
    return masked

fault_spectrum = [[0.3] * 12, [0.8] * 12]
masked_spectrum = safe_time_mask(fault_spectrum, 1, 4, 7, 10)
assert all(row[1:4] == [0.0, 0.0, 0.0] for row in masked_spectrum)
assert fault_spectrum[0][1] == 0.3
try:
    safe_time_mask(fault_spectrum, 8, 11, 7, 10)
except ValueError:
    pass
else:
    raise AssertionError("event-overlapping mask was accepted")

Performance and operating cost

The reference copies a frequency-by-time array, costing O(FT) time and space for F bins and T frames; an in-place tensor operation can avoid the copy but must not mutate cached validation features. Mask generation itself is O(FM) for mask width M when applied to every frequency row. The real operating cost is extra training variants and label review. A mask that removes the event can lower model quality despite being computationally cheap.

Common Mistakes

Do not apply augmentation masks to the final clean evaluation without labeling it a stress test.
Do not let a time mask erase the only positive event.
Do not assume zero means silence after log scaling or standardization.