8

I'm having an issue here where I try to automate a setup with Ansible.

Some of the steps require interaction with apt, but occasionally I get an error because unattended-upgrade kicked off and locked apt. This will make the playbook stop.

I've tried many ways around this, the most successful being the repetition of a failed apt command.

But this does not scale, is also not 100% reliable and feels bad.

I've opted to issue an apt -y purge unattended-upgrades right at the beginning of the playbook. I also tried apt -y remove unattended-upgrades, but that one seems to return while it is still at work. Purging appears to shut down unattended upgrades as before it exits, which is what I want.

But it turns out that even that call to apt -y purge unattended-upgrades can fail due to locking. So I changed it to while [[ $(dpkg -l | grep -P "unattended-upgrades" | wc -c) -ne 0 ]]; do apt -y purge unattended-upgrades; done, but also that fails occasionally (I can't figure out why)

I need one command which, when executed, will terminate and bury unattended upgrades immediately, regardless if it is running or not, and make the guarantee that it won't start anymore as soon as that command returns, until I explicitly apt install it again. It is ok if that command takes a minute to finish it's job.

Also, the system doesn't have Python installed, so Ansible is only issuing raw commands, until I manage to install Python which should be after a successful call to apt -y update

I am in a state where I can trigger unattended upgrades easily, since this is a VM, and as soon as I issue a date -s command to correct the stale date, unattended-upgrade kicks in. After starting the VM, I have a couple of minutes until date corrects itself automatically which then starts unattended-upgrades.

This is what I'm doing now:

- name: Disable autoupdate (part 1 of 2)
  raw: sed -i /Update/s/"1"/"0"/ /etc/apt/apt.conf.d/10periodic && sync

- name: Disable autoupdate (part 2 of 2)
  raw: echo 'APT::Periodic::Unattended-Upgrade "0";' >> /etc/apt/apt.conf.d/10periodic && sync

- name: Terminate any active autoupdate
  raw: ps -A | grep unattended-upgrades | awk '{print $1}' | xargs -r kill -15 $1

- name: Terminate any active dpkg
  raw: ps -A | grep dpkg | awk '{print $1}' | xargs -r kill -15 $1

- name: Allow dpkg to recover
  raw: dpkg --configure -a

- name: Purge autoupdate
  raw: apt -y purge unattended-upgrades    

- name: Update apt cache
  raw: apt -y update

- name: If needed, install Python
  raw: test -e /usr/bin/python || apt -y install python

Terminating dpkg is what creeps me out. All that is run on a fresh install of Ubuntu Server 18.04.1


Here is the solution created by using the accepted answer:

https://stackoverflow.com/a/51919678/277267

13
  • 1
    I never allow unattended-upgrades near a production server. However, I do advise doing all updates before running other ansible operations. Aug 19, 2018 at 13:28
  • 1
    Is there a benefit of using dpkg --purge unattended-upgrades instead of apt -y purge unattended-upgrades (which I'm doing now)?
    – Daniel F
    Aug 19, 2018 at 13:42
  • 4
    btw you're alledgedly supposed to use apt-get in scripts, so that the interactive apt program has full freedom to change behaviour to make it more friendly.
    – sourcejedi
    Aug 19, 2018 at 13:44
  • 1
    @RuiFRibeiro Error due to Ansible defaulting to python, ubuntu1804 only has python3. linuxconfig.org/… / docs.ansible.com/ansible/latest/installation_guide/…
    – sourcejedi
    Aug 19, 2018 at 13:58
  • 1
    @RuiFRibeiro not sure how much real Ansible makes a difference in this situation. github.com/ansible/ansible/issues/25414
    – sourcejedi
    Aug 19, 2018 at 14:02

1 Answer 1

12

Apparently unattended-upgrades is run from one of the systemd units apt-daily.service / apt-daily-upgrade.service. (These in turn are triggered by systemd .timer units, with the same names).

You could try to wait for the systemd units as follows:

systemd-run --property="After=apt-daily.service apt-daily-upgrade.service" --wait /bin/true

This is independent of whether you want to send SIGTERM to dpkg or apt-get or something to try and get them to finish quicker. kill only transmits a signal; it does not wait for anything. In principle you always need some way to wait, before you can use the resource that is freed up.

4
  • What does that command do? Wait until dpkg finishes if it has been started due to an unattended upgrade? I'd prefer doing that instead of sending a SIGTERM to it, which is what I'm doing now.
    – Daniel F
    Aug 19, 2018 at 13:43
  • It assumes systemd (and new enough apt). It waits for the apt timer-triggered services to complete, if they are running. Apparently those are what run the scheduled unattended-upgrades jobs. Maybe it's a more general approach, since apt-daily might be running a daily apt-get update even if the unattended-upgrades package is not installed. I don't know anything about killing the unattended-upgrades process beforehand - no idea whether that does anything useful, or is any less brutal than killing dpkg.
    – sourcejedi
    Aug 19, 2018 at 13:49
  • 1
    So first disabling autoupdate via /etc/apt/apt.conf.d/10periodic modification and then issuing your command and waiting for it to finish should leave the system in a state where only I am triggering dpkg (through whatever means)? I'll give it a try.
    – Daniel F
    Aug 19, 2018 at 13:54
  • 2
    I just want to link here my complete solution which is based on your answer stackoverflow.com/a/51919678/277267
    – Daniel F
    Aug 19, 2018 at 16:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .