Backfill prompts: account for writes during migration

Last updated: 4 Oct 202611 min read

tutorial

A live backfill must account for changes after its snapshot through a write pause, controlled dual write, or ordered change-stream replay. The prompt should ask which system owns writes, how updates and deletes are represented, what watermark marks a complete tail, and how lag is observed. A zero-lag screenshot is not a durable completion claim if more events can arrive. The cutover plan needs an explicit stable gate and divergence monitoring after traffic changes.

Operational case

Harbor leaves the legacy table as write owner during snapshot H-47. New order updates enter an ordered tail that the target replays after backfill, including changes to quarantined IDs. The operator records the last applied watermark and measured lag at the planned cutover window. A generated paragraph cannot attest that the live stream is caught up; the running system supplies that evidence.

Write owner: legacy table
Snapshot: H-47 | tail: ordered positions
Update/delete: stable order key + source version
Cutover gate: measured lag and replay watermark

Performance and review cost

Applying T tail events is O(T) expected keyed work after O(N) snapshot processing. Stream ordering and deduplication may add cost. Controlled dual write can introduce partial-failure states, so any selected strategy needs a recovery path and observable lag receipt.

Common Mistakes

Do not ignore post-snapshot updates.
Do not give two systems write ownership without a conflict rule.
Do not claim the tail is complete without a watermark.

Connected lessons

prompt engineering

database migration