Skip to content
AITroveRead. Build. Understand.
Reading list
Make this comfortable

CI matrix gates: distinguish skipped, canceled, experimental, and passed

Last updated: 5 Oct 20266 min read
tutorial
AdvancedBy AITrove Editorial

A matrix expands one CI job into several platform or runtime variants. A green result for one variant does not establish that all supported variants passed. Cancellation from fail-fast, conditional skips, and nonblocking experimental rows can change what a summary check means. A release gate must state which matrix entries are required, how missing entries are detected, and when an experiment may fail without blocking.

Operational decision

A receipt API ships on two operating-system architectures. Run the same contract suite on both supported runners and record each result by architecture. The fragment disables fail-fast so one failure does not erase evidence from the other row; it does not itself implement the final gate. Build a stable required summary job that waits for every supported row, rejects failure, cancellation, and absence, and publishes the revision and image digest it checked. Keep experimental rows separate from that summary rather than marking all failures nonblocking. Deliberately break one architecture, cancel another, and skip a row with a condition in a test repository; confirm the protected branch cannot accept any of those as a complete supported-platform pass. Compare matrix rows with the architectures actually present in the promoted image index. A green CI matrix on an architecture that is never deployed is less useful than a runtime test on every production target.

yaml
name: receipt-platform-check
on:
  pull_request:
  merge_group:
    types: [checks_requested]
jobs:
  platform-test:
    strategy:
      fail-fast: false
      matrix:
        runner: [ubuntu-24.04, ubuntu-24.04-arm]
    runs-on: ${{ matrix.runner }}
    steps:
      - uses: actions/checkout@v4
      - run: ./ci/test-receipt-contract.sh

Cost and verification

A full matrix multiplies runner minutes and can lengthen queue time; disabling fail-fast spends more after the first failure but preserves diagnostic evidence. The summary job adds a small amount of orchestration and needs careful handling of missing checks. Measure per-row duration, cancellation rate, supported architecture coverage, and differences between tested and shipped images. Runner labels and availability are environment-specific, so validate them against the repository's actual hosted or self-hosted runner pool before adopting the fragment.

Common Mistakes

  • Do not count a skipped or canceled supported row as passed.
  • Do not mark a required architecture experimental to keep releases moving.
  • Do not claim runtime coverage from a build-only matrix.

Connected lessons

Practice and check

devops
operations
Storage details