Configuration management: converge hosts without surprise restarts

Operational decision

For a report-renderer host, manage its configuration file from a reviewed template and notify a restart handler only when that file changes. Validate the rendered configuration before a restart, and roll out to one host before the rest of the fleet. Ansible check mode can preview many module changes, but some tasks cannot predict their result or may be skipped; it is not a substitute for a disposable-host test. The play below uses a managed template and one handler. It assumes the template exists in the role and that the service name matches the host package. Store credentials in a secret system, not the template repository. Record the inventory version and deployment result so a host drift report points back to the same reviewed change.

- name: Configure report renderer
  hosts: renderers
  become: true
  tasks:
    - name: Install service configuration
      ansible.builtin.template:
        src: report-renderer.conf.j2
        dest: /etc/report-renderer.conf
        mode: '0640'
      notify: Restart report renderer
  handlers:
    - name: Restart report renderer
      ansible.builtin.service:
        name: report-renderer
        state: restarted

Cost and verification

A host-wide run can consume network and package-manager capacity, so use batches and a rollback plan. A handler avoids restarting an unchanged service, yet a changed file can still trigger a disruptive restart; use the application's reload or drain behavior where available. Template output can contain sensitive values even when the source file does not. Limit diff output and retention if secrets may be rendered. Track failed hosts explicitly rather than reporting fleet success from only the first completed batch.

Common Mistakes

• Do not use an append-only shell command for a convergent setting.
• Do not assume check mode predicts every task.
• Do not restart all hosts at once without capacity headroom.

Connected lessons

• DevOps: delivery, infrastructure, and reliable operations
• Linux service diagnostics: process, socket, and journal
• Terraform state: shared ownership and safe plans
• Incident response: contain impact, then learn