Skip to content
AITroveRead. Build. Understand.
Reading list
Make this comfortable

Project: migrate a dashboard with shadow reads

Last updated: 6 Oct 20265 min read
project
AdvancedBy AITrove Editorial

Move a regional payments dashboard to a second query engine while preserving its keys, exact monetary totals and time-bucket behavior.

Freeze the input and contract

Pin a source generation containing 470 payments across three regions, null customer references, a correction and events close to a UTC midnight boundary. State the dashboard grain, minor-unit money representation, permitted filters and expected report time zone. Save both engine session settings. Semantic probes must pass before a production-sized shadow run.

Translate and run unseen

Translate the detail and aggregate queries to the target dialect. Execute them under a separate shadow identity, without displaying target answers to users. Attach source and target generation IDs to each result. Limit replay to a fixed concurrency and scan budget. Compare stable keys and exact cents totals, then measure p50 and p95 latency for the same filter shapes.

Plant three faults

Make the target treat one null reference as zero, shift one event into the next local day and duplicate a corrected payment. Show that row-count-only and grand-total-only checks each miss at least one planted fault. The discrepancy report must identify affected keys, categories and query revisions. Shadow triage should produce a small reproducible fixture for each cause.

Gate and canary

Repair the translation, replay multiple intervals and require exact key coverage and exact cents totals for critical queries. Move only one dashboard cohort to the new engine. Observe freshness and latency while the old route remains available. If one later correction diverges, roll the cohort back to the prior pointer and keep the failed candidate for diagnosis.

Deliver evidence

Submit the contract, source generations, SQL revisions, semantic fixtures, mismatch reports before and after repair, spend limit, query latency and canary rollback trace. Include one unresolved scenario and mark it as blocking; a cutover gate should not silently ignore a missing interval. The final dashboard result should be reproducible from the pinned manifest and comparison inputs.

Implementation

python
from collections import Counter

source_payments = [("pay-47", "west", 2375), ("pay-48", "east", 6400),
                   ("pay-49", "west", -125)]
target_payments = [("pay-49", "west", -125), ("pay-47", "west", 2375),
                   ("pay-48", "east", 6400)]

def reconcile(records):
    keys = Counter(payment_id for payment_id, _, _ in records)
    totals = {}
    for payment_id, region, cents in records:
        totals[region] = totals.get(region, 0) + cents
    return keys, totals

assert reconcile(source_payments) == reconcile(target_payments)
assert reconcile(source_payments)[1] == {"west": 2250, "east": 6400}

Performance and operating cost

The reference reconciliation uses O(N) expected time and O(K + G) space for N rows, K distinct payment keys and G regions. Real shadow reads add two engine scans, comparison storage and a bounded canary period. Partitioned hashes can reduce detailed comparisons, but every mismatch must be resolved against pinned row evidence before a critical dashboard is moved.

Common Mistakes

  • Do not expose the shadow result before its gate passes.
  • Do not accept aggregate totals without checking duplicate and missing payment keys.
  • Do not lose the old routing pointer before the canary interval ends.

Read next

ai-data
data-engineering
Storage details