Project: prove a receipt release passed the checks that matter

Last updated: 5 Oct 20269 min read

project

Use an isolated receipt repository with a payout dependency, two supported build targets, a disposable database, and a test registry. Configure a protected branch and merge queue in that test repository. Preserve the candidate revision, selected checks, runner identity, service image digest, test report, and deployment selection for every experiment. The drill should fail when evidence is missing, not merely when code has a syntax error.

Break the gate deliberately

Make two pull requests pass alone but fail as a combined dependency change; prove the merge-group check rejects their candidate. Edit a shared schema and show that the selective selector runs the full suite. Remove the base revision from a shallow checkout and verify the same safe fallback. Introduce a parallel test-data collision, then isolate database, queue, cache, and object scopes by run. Reproduce one intermittent failure, place only that test in a time-bounded nonblocking lane, and retain a blocking settlement smoke check. Simulate a skipped required workflow and a canceled architecture row; neither may count as a completed release gate.

Receipt release evidence gates
Merge candidate: checked revision equals accepted revision
Selection: shared change and unknown base run full suite
Quarantine: named test, owner, expiry, replacement coverage
Isolation: two parallel runs share no mutable records
Inputs: clean runner and declared service versions
Matrix: all supported rows completed and passed
Artifact: tested digest equals promoted digest
Rebuild: differences explained before reproducibility claim

Bind checks to release bytes

Build and push one image, pull it by digest for final tests, then submit the same digest to a simulated deployment gate. Retag the repository between test and promotion; the gate must still choose the approved digest. Build the package twice from clean runners with fixed inputs and compare both whole-file digests and first differing archive member. Do not declare a mismatch hostile without investigation, and do not declare two matching packages safe without knowing who produced them. Keep the evidence tied to the exact revision and builder.

Cost and verification

Record queue delay, full-suite fallback frequency, quarantine age, orphaned test scopes, cold-run time, matrix runner minutes, and registry storage through rollback. The project passes only when a bad combined commit, missing supported row, or changed image digest blocks promotion. A dashboard showing green from one earlier pull-request head is insufficient.

Common Mistakes

Do not convert a skipped workflow into a passing gate.
Do not rerun a flaky suite until it becomes green and discard the first failure.
Do not rebuild the image after final tests.

Connected lessons

devops

project