Retention is a data-flow property, not a single database TTL. The same case note may exist in primary rows, search documents, image variants, browser caches, analytics events, export artifacts, worker messages, and backups. Start with an inventory naming each store, the purpose of its copy, owner, access scope, and approved retention period. Policy and law set the periods; implementation should not invent them. Each expiring copy needs a deletion or irreversible de-identification path, and derived stores need a way to learn that the source record changed or expired. Keep a small non-sensitive completion record so operators can prove the workflow ran without retaining the deleted content.
Make this comfortable
Retention Inventory and Expiry Workflows
Working case
Case 47 reaches an approved retention boundary. The primary row is removed, but its title remains in the search index and a 740 MB export persists in object storage. A reviewer can no longer open the case page yet still finds it in autocomplete. The expiry workflow first prevents new access, records the target and policy version, queues cleanup for search, media, exports, and caches, then verifies each target. A failed index deletion becomes a visible retryable task rather than a quiet success. Backup expiry follows a separate schedule and restore process.
Implementation boundary
function expiredAt(now, deadline) {
return Number.isFinite(deadline) && now >= deadline;
}
console.log(expiredAt(93, 62));
// Output: trueThe helper only decides a deadline. A production inventory should list every copy and specify how a record key maps to it. Run expiry with a stable deletion operation ID; retries must not recreate a record or treat missing rows as failure when they were already removed. Prevent reads during multi-store cleanup with a tombstone or access gate, then fan out bounded cleanup tasks. Verify each store and keep a minimal outcome record with no original note. If a store cannot delete promptly, document its restricted access and deadline. Ensure new search-index rebuilds and exports honor tombstones, not only live row absence.
Cost and boundaries
Scanning n records for expiry is O(n) without an index or scheduled partition; an indexed deadline query can fetch due rows proportionally to the due set. Each deletion fans out to k derived stores, making total work O(k) per record before backup handling. Keeping tombstones uses storage but prevents reappearance during asynchronous cleanup and restore. Aggressive polling raises load; large intervals extend the period between deadline and completed cleanup. Measure due backlog, oldest overdue item, retry count, and per-store completion rather than only counting primary-row deletions.
Failure trace
A database TTL removes case 47. The search index still returns its title and an old export remains downloadable. The team considered deletion complete because the primary query returns 404. Define completion across stores and gate reads while cleanup proceeds. Another failure lets a nightly index rebuild replay old change events after a deletion, resurrecting a searchable document. Carry a deletion generation or tombstone through the rebuild and test replay order. Do not put full private content into the cleanup log just to make the operation auditable.
Verification
- Every active copy has an owner, purpose, retention decision, and cleanup path.
- An index or artifact failure cannot be reported as complete expiry.
- Rebuilds cannot revive a tombstoned record.
Practice drill
Inventory every location reached by a case note and its attachment. Expire case 47, deliberately fail search deletion, and confirm the case is inaccessible while the cleanup task remains visible. Retry the task, rebuild the index from a stale event stream, and check that the document does not return. Inspect the export artifact and browser cache paths. Run the same operation twice and verify it is safe. Report completion only after every active derived store meets its policy, while recording backup handling separately.
Decision note
Treat retention as a mapped multi-store workflow with read gating, retries, and store-level evidence.
Common Mistakes
- Equating primary-row deletion with complete expiry.
- Ignoring derived search and export copies.
- Writing the deleted private content into cleanup evidence.
Connected lessons
Data Retention, Export, and Erasure; Export and Erasure Job State; Backup Restore with Deletion Tombstones; Browser Storage Cleanup on Account Change; Search Index Freshness and Rebuilds; Telemetry Minimization and Retention; Background Jobs and the Outbox Boundary.
Apply and check
Build Project: verified data expiry and review Web Development: authorization and data lifecycle decisions quiz.
Further connections
Private Asset Downloads, Revocation, and Expiry.
Further connections
web-tech
web-development