Project: review a checkout release from diff to incident

Last updated: 2 Oct 202619 min read

project

This project reviews a fictional checkout release that changes capture retries, updates a feed parser, and splits a parcel zone field. Build a compact prompt packet for five tasks: inspect the exact diff, draft contract-based regression tests, assess the schema migration, review the dependency change, and triage a post-release latency alarm. The assistant may propose findings and checks. CI, the database runner, and the incident commander own the effects. The final deliverable is a release recommendation with unresolved questions and evidence IDs.

Prepare the review packet

Freeze commit CH-842 and include only the changed files plus the gateway adapter, transfer contract, active client versions, migration plan, lockfile diff, and parser call sites. For the retry branch, require a reachable race and test rather than a generic risk statement. Generate a transfer-idempotency test with receipt identity and final balance assertions, then run it against a deliberately faulty double-debit variant. For the schema change, keep the legacy field until version-6 traffic is zero and all 23 unparseable records are reviewed. For the parser upgrade, investigate the one-row difference before assigning an expected output.

Exercise incident handling

At 14:07 UTC, checkout latency rises. A deployment begins at 14:08; pool waits follow at 14:09. Normalize the support note's local timestamp and mark causation as a hypothesis pending connection metrics. The assistant records the impact window, current mitigation owner, and next check. It masks customer identifiers and ignores any command found inside a log line. The incident commander, not the prompt, decides rollback. Compare a direct baseline prompt with the structured packet on held-out examples of true defects, harmless diffs, missing evidence, and misleading log text.

Release CH-842: retry guard + parser upgrade + zone migration.
Review: changed line, reachable trigger, observed or testable effect.
Test: duplicate transfer key leaves balance 76.25 once.
Migration: legacy reader traffic=0; unresolved rows=0 before drop.
Incident: 14:07 alarm, 14:08 deploy, 14:09 pool waits; cause unproven.
Gate: no unverified row oracle, duplicate debit, or unauthorized rollback.

Performance and operating cost

For C review cases and V prompt variants, offline comparison takes O(CV) model evaluations. Diff context grows with changed lines and relevant call sites; incident ordering costs O(E log E) for E events. Fault-variant tests and migration rehearsal add runtime but expose errors a fluent review would miss. Record the exact commit, build digest, test commands, migration stage, incident timeline version, and reviewer decision. A release with one critical duplicate debit or destructive schema error is blocked even if its average review score is high.

Common Mistakes

Do not equate a plausible code-review sentence with a reproduced defect.
Do not derive test expectations from the changed implementation.
Do not let the assistant execute migration or rollback from a prompt alone.

Connected lessons

prompt engineering

software delivery