A LUKS volume depends on its encrypted data area, header and keyslots, and at least one valid way to unlock a slot. A header backup can rescue certain metadata damage, but it is sensitive material: together with a passphrase valid when it was made, an older backup may still unlock data after that passphrase is removed from the live header. A header backup does not replace a data backup, and restoring an old header can undo later keyslot changes. Keep recovery credentials and header backups under separate, audited controls. Practice against a copied disposable volume, never the only production disk.
LUKS recovery drills: preserve independent unlock and header recovery paths
Operational decision
A batch-processing host uses one automated boot unlock path and one sealed recovery passphrase. Before rotating the automated secret, an operator verifies the recovery slot on a cloned encrypted volume and captures a protected header backup. They add and test the new slot, then remove the retiring slot only after a complete reboot of a disposable clone proves the new boot path. A second drill deliberately corrupts the clone's header and restores the matching protected backup, then verifies file hashes from a separate data backup. The drill records who accessed the recovery material, how long the host stayed unavailable, and whether an older header backup could still admit a retired credential.
Volume inventory: batch-worker-root, clone only
Recovery slot: separate sealed passphrase
Header backup: protected and versioned with keyslot changes
Proof: cold unlock, reboot, file-hash comparison
Forbidden test target: sole production copyCost and verification
Independent recovery material costs storage, access review, and periodic testing, but prevents a single broken automation path from stranding an encrypted host. Each header backup increases the set of credentials that may recover old keyslots; destruction and retention therefore belong in the rotation plan. A successful unlock test proves access to a volume, not integrity or completeness of its files. Pair it with a data restore and application-level verification. If the recovery passphrase is stored in the same inaccessible volume, the apparent second path is circular.
Common Mistakes
- Do not restore an old header onto the sole production volume as a practice test.
- Do not treat a header backup as a backup of application data.
- Do not forget that an old header backup can preserve access for a retired passphrase.
Connected lessons
- DevOps: delivery, infrastructure, and reliable operations
- Secret rotation rollout: update the issuer, consumer, and active connections
- Recovery drills: retain evidence and reset the next line of defense
- Break-glass access: recover control without permanent privilege
- Linux kernel rollouts: prove the running kernel after each reboot cohort
- SSH host-key rotation: change server identity without teaching clients to ignore warnings
Practice and check
- Linux host change drill: reboot, access, firewall, and disk recovery
- Linux host change decisions: access and recovery quiz
