Online feature serving: materialization, staleness and fallback

Last updated: 5 Oct 20265 min read

tutorial

Online retrieval should return a versioned value with its age and a defined response when the feature is absent or stale.

Separate stores by workload

Historical retrieval scans past entity states for training; online serving needs low-latency reads for current decisions. Materialization copies selected feature values to a serving store under a known cutoff. This is a second publication step, not proof the values are fresh. An atomic publication boundary prevents a partly updated batch from appearing as a complete snapshot.

Bound age and latency

Set an age limit for each feature based on its decision role. A lesson catalog attribute may tolerate a longer delay than a current fraud signal. At request time, log feature version, value timestamp, age, missing flag and retrieval latency. If the store times out, use a documented fallback or decline the decision; never substitute a silent zero for an unknown risk count.

Avoid partial feature vectors

A model expecting features from one coherent snapshot can behave unpredictably when one key has the new materialization and another has the old one. Use snapshot IDs or compatible per-view versions in the request contract. If mixed versions are permitted, test them explicitly. Training-serving parity should include timestamp and default behavior, not merely value equality.

Run a stale-key fixture

Merchant M has a value 7 materialized at minute 120, and the decision arrives at minute 173 with a maximum age of 47 minutes. The value is stale. A newly registered merchant has no row; return missing with fallback metadata. Simulate one serving timeout and verify the request trace distinguishes missing, stale and unavailable states.

Implementation

def online_lookup(feature_by_entity, entity_id, now_minute, max_age_minutes):
    record = feature_by_entity.get(entity_id)
    if record is None:
        return "missing", None
    age = now_minute - record["materialized_minute"]
    if age < 0:
        return "invalid-time", None
    if age > max_age_minutes:
        return "stale", None
    return "ready", record["value"]

Performance and operating cost

A hash-map lookup is O(1) expected time and O(1) request space, while the online store retains O(E) entity records. Network latency, serialization and cache invalidation dominate a real request; measure tail latency and stale-key rate.

Common Mistakes

Do not equate successful lookup with fresh data.
Do not collapse missing, stale and timeout into the same zero default.
Do not publish a partly refreshed vector without a compatibility rule.