Secrets and configuration across the delivery path

Operational decision

A settlement worker reads a database credential from a restricted runtime secret source rather than baking it into the image. Give the workload only the secret it needs, protect the cluster store with encryption at rest and access control, and avoid printing environment variables during troubleshooting. Rotate a credential by introducing the new value, verifying connections, then revoking the old one; abrupt replacement can strand long-lived connections. Use separate identities for CI, deployment, and application runtime. The YAML fragment shows a key reference, not the secret value or its creation. Whether an update reaches a running process depends on the injection method and application behavior. Test revocation and incident response, not just initial access.

env:
  - name: SETTLEMENT_DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: settlement-db
        key: password

Cost and verification

A managed secret service and frequent rotation add requests and operational work, but reduce the blast radius of a leaked credential. Environment variables are convenient yet can be exposed through process inspection or accidental diagnostics; a mounted file or direct secret client may fit stricter environments. No storage mechanism prevents an authorized process from mishandling the secret. Restrict log access and set retention. Check that backups and disaster-recovery copies protect the same material.

Common Mistakes

• Do not commit base64-encoded credentials.
• Do not share one cloud identity between CI and the running service.
• Do not rotate a credential without testing live connection behavior.

Connected lessons

• DevOps Tutorial
• GitHub Actions: narrow tokens and cloud trust
• Terraform state: shared ownership and safe plans
• Backups and disaster recovery: prove the restore path

Operational follow-up

• Kubernetes NetworkPolicy: permit only required flows
• Kubernetes RBAC: bind one service account to one job

Advanced follow-up

• Secret rotation rollout: update the issuer, consumer, and active connections

Advanced follow-up

• Encryption key rotation: keep old data decryptable during recovery

Advanced follow-up

• ConfigMap projection: prove when a running process sees a new value
• Required configuration keys: fail startup before accepting traffic
• Secret access: audit workload creation as an indirect read permission
• Kubernetes Secret encryption: rotate keys without losing restore access

Web publishing follow-up

• Preview and production publishing: keep content and configuration separate

Related Prompt Engineering lesson: Prompt privacy: send only the fields needed for the task.