Kustomize overlays: review the complete environment diff

Operational decision

A claims-processing service has development and production overlays. The base adds a new container port, while production changes the image digest and resource requests. The CI job renders both overlays, validates every output object against the cluster API version, and computes a structured diff from the last accepted production render. It flags a selector change because that can orphan the current Pods, a new ServiceAccount because it changes identity, and a generated ConfigMap name because a rollout may follow even if the original literal did not change. Run a negative test in which a patch target no longer matches after a base rename; the pipeline must fail or clearly report the missing intended modification, not silently deploy the unpatched base. Avoid patching a shared base with production-only secrets. For each change, reviewers see the affected clusters, resource identities, image digests, and relevant fields, then verify that the controller renders the same output. A successful YAML parse is insufficient if the overlay points at the wrong namespace or strips an admission label.

resources:
  - ../../base
namespace: claims-production
images:
  - name: registry.internal/claims-processor
    digest: sha256:7e3a448df91e0c641fd37435a7c78890db03b9fd625126995dc975fb4d88727a
patches:
  - path: resources-production.yaml
    target:
      kind: Deployment
      name: claims-processor

Cost and verification

If E environments each render N resources, full rendering and policy validation costs roughly O(E × N) objects plus template and diff work. Selective execution can be safe only when the dependency graph from bases to overlays is complete; otherwise a shared-base edit may skip production. Track unreviewed rendered changes, missing patch targets, unexpected deletions, and differences between CI output and controller output. Cache by immutable input digest, never by a movable branch name alone.

Common Mistakes

• Do not review only the overlay patch while ignoring the rendered base.
• Do not let a nonmatching patch silently remove a production constraint.
• Do not skip production checks after a shared-base change.

Connected lessons

• DevOps: delivery, infrastructure, and reliable operations
• GitOps reconciliation: desired state and drift
• Helm release review: render before applying
• Immutable configuration: bind each release to one named generation
• CI matrix gates: distinguish skipped, canceled, experimental, and passed

Practice and check

• Project: test GitOps render, ownership, prune, and rollback boundaries
• DevOps GitOps operating-boundary decisions