2

Using ansible often relies on a property called "idempotence". If you apply a role a second time, it's expected to have the same result. E.g. it won't add a second copy of the same line in a config file.

There's another property you might call "resumable". In the event of a network partition between controller and target, can an interrupted play be re-run when the partition is resolved?

Ansible hints towards this property, by listing failed targets & allowing them to be re-tried.

However if you review the practical examples, they do not appear to hold this property. Suggest an approach which will ensure I recognize any failures of my playbooks to resume correctly.

- name: Create Mysql configuration file
  template: src=my.cnf.j2 dest=/etc/my.cnf
  notify:
  - restart mysql

- name: Start Mysql Service
  service: name=mysqld state=started enabled=yes

E.g. the above snippet from ansible-examples, fails to achieve the "resumable" property. The handler "restart mysql" will never trigger, if the play is interrupted after creating the config file but before running the handler "restart mysql".

0

In principle, resumable plays could be implemented by touching a timestamp file after restarting the service. Then the condition for restarting the service is whether the restart timestamp is before the configuration file's modification time. (Inspired by make).

In the case of configuration files, this is also possible using native ansible modules:

- name: Create Mysql configuration file
  template: src=my.cnf.j2 dest=/etc/my.cnf

- name: query  | mysql has been restarted with new config file
  template: src=my.cnf.j2 dest=/ansible-managed/mysql/restarted/my.cnf
  check_mode: yes
  register: mysql_restarted

- name: ensure | mysql has been restarted with new config file
  service: name=mysqld state=restarted
  when: mysql_restarted|changed    

- name: record | mysql has been restarted with new config file
  template: src=my.cnf.j2 dest=/ansible-managed/mysql/restarted/my.cnf

(or e.g.

- name: query  | mysql has been restarted with new config file
  copy: remote_src=yes src=/etc/my.cnf dest=/ansible-managed/mysql/restarted/my.cnf
  check_mode: yes
  register: mysql_restarted

although remote_src will only work with individual configuration files, not directories)

0

Ansible docs mention this but only in passing. "Certain errors could still prevent the handler from running, such as a host becoming unreachable." There are no suggestions about what to do in this case.

It mentions --force-handlers. There is some confusion here. Issue #4777 requests a --force-handlers option which would run all handlers, regardless of whether they were notified, to allow recovery in this situation. This issue was closed with the comment "this is now implemented". Narrator: it was not implemented. I have opened a new issue to request such a feature.

Unfortunately I believe this comment has caused some to suggest --force-handlers as the solution to this problem on stackexchange or elsewhere, when it is not.


A full solution would modify ansible to record a database of pending handlers. (The handler would be recorded immediately when a task detects a change is about to be made).

Short of that, you would want to avoid such interruptions. Because you would need to carefully review both handlers, and any tasks conditional on |changed, and make sure to run all of them on the retried targets.

Moral: it might be useful to use handlers, and avoid sprinkling |changed throughout a playbook.

A less elegant but equally correct solution would involve modifying ansible with an option to force running all handlers. Or perhaps to treat all non-skipped tasks as changed. If the original run needed to restart the service, it's not completely unreasonable to schedule a second run which restarts the service. The disadvantage is if the original run only needed to restart some services.

You could also avoid interrupted plays by using ansible "pull mode". Or, if the main concern is because ansible is run remotely, you could run ansible on a server in the same site, using a persistent session with screen / tmux.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.