2

I have a very simply stansa in my cloud-init,

write_files:
  - path: /home/ecarroll/.zshrc
    content: |
      source ~/powerlevel10k/powerlevel10k.zsh-theme
    owner: ecarroll:ecarroll
    permissions: '0644'
    defer: true

The defer flag is documented in the Cloud Init: Write Flags document,

defer: (boolean) Defer writing the file until ‘final’ stage, after users were created, and packages were installed. Default: False.

But when I use it as-above, I get

schema.py[WARNING]: Invalid config: write_files.0: Additional properties are not allowed ('defer' was unexpected)
util.py[DEBUG]: Writing to /home/ecarroll/.zshrc - wb: [644] 47 bytes
handlers.py[DEBUG]: finish: init-network/config-write-files: FAIL: running config-write-files with frequency once-per-instance
util.py[WARNING]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>) failed
util.py[DEBUG]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>) failed

The actual reason why I need defer is found in the trace. It's because I need the file to be chown'd to the user (ecarroll) that cloud-init has not yet created,

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1380, in chownbyname
    uid = pwd.getpwnam(user).pw_uid
KeyError: "getpwnam(): name not found: 'ecarroll'"

I am on Ubuntu 20.04.4 LTS.

2 Answers 2

3

You need to update to cloud-init v21.4, which is the version that added the defer property to write_files.

Check the version you are running with

  • cloud-init -v, and with
  • head -n 1 /var/log/cloud-init.log see Brett's answer above

If these two disagree, you may want to check out Why does the cloud-init version in log file conflict with the version in the binary?

1
  • 1
    When I run that, I see /usr/bin/cloud-init 21.4-0ubuntu1~20.04.1 which would incline me to believe I am on 21.4? Mar 9, 2022 at 22:52
2
+250

What does the following show?

head -n 1 /var/log/cloud-init.log

Without the defer statement I see a similar exception on Ubuntu 20.04 cloud-init 22.1:

arc~ cat ~/my-user-data
#cloud-config
users:
  - default
  - name: ecarroll
write_files:
  - path: /home/ecarroll/.zshrc
    content: |
      test content
    owner: ecarroll:ecarroll
    permissions: '0644'
arc~ lxc init ubuntu:focal ecarroll  -c user.user-data="$(cat ~/my-user-data)"
Creating ecarroll
arc~ lxc start ecarroll
arc~ lxc exec ecarroll -- grep -A12 Traceback /var/log/cloud-init.log
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1430, in chownbyname
    uid = pwd.getpwnam(user).pw_uid
KeyError: "getpwnam(): name not found: 'ecarroll'"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 976, in _run_modules
    ran, _r = cc.run(run_name, mod.handle, func_args,
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 186, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 205, in handle
    write_files(name, filtered_files)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 245, in write_files
    util.chownbyname(path, u, g)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1434, in chownbyname
    raise OSError("Unknown user or group: %s" % (e)) from e
arc~ lxc exec ecarroll -- grep "Additional properties" /var/log/cloud-init.log
arc~ lxc stop ecarroll && lxc rm ecarroll   

If I add defer, it works as expected:

arc~ cat ~/my-user-data
#cloud-config
users:
  - default
  - name: ecarroll
write_files:
  - path: /home/ecarroll/.zshrc
    content: |
      test content
    owner: ecarroll:ecarroll
    permissions: '0644'
    defer: true
arc~ lxc init ubuntu:focal ecarroll -c user.user-data="$(cat ~/my-user-data)" 
Creating ecarroll
arc~ lxc start ecarroll                                                      
arc~ lxc exec ecarroll -- grep -A12 Traceback /var/log/cloud-init.log         
arc~ lxc exec ecarroll -- grep -A12 "Additional properties" /var/log/cloud-init.log
arc~ lxc shell ecarroll                                                            
root@ecarroll:~# cat /home/ecarroll/.zshrc 
test content
root@ecarroll:~# exit
logout
arc~ lxc stop ecarroll && lxc rm ecarroll

Note that the "Additional properties" warning does not exist in my log. This warning indicates that you've used a key in your cloud-config that the jsonschema validator doesn't know about. In this case that indicates that the running version predates the defer keyword.

Note that in the traceback that you reported, getpwname is on line 1380.

The most recent version of cloud-init on 20.04 that this is true for is release 21.2.

By release 21.3, that call to getpwname has been pushed back to line 1429.

Based on the above information, I would assume that cloud-init ran at a version that didn't support defer and was upgraded afterwords to 21.4.

If you want to try rerunning all the cloud-init stages, you can always try a cloud-init clean --logs followed by a reboot. This should remove cloud-init artifacts. You may find the lxc commands above are helpful for trying out modules locally prior to running in a cloud.

4
  • 2022-01-21 18:03:44,127 - util.py[DEBUG]: Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'init-local' at Fri, 21 Jan 2022 18:03:44 +0000. Up 48.49 seconds. Now I'm really confused.. Mar 17, 2022 at 14:28
  • Asking a follow up question on this. looking at the head -n1 of the file was way insightful. Mar 17, 2022 at 14:29
  • That's what I figured :) I'm not sure how/why your image started with an old version of cloud-init, but that explains the behavior. Mar 17, 2022 at 14:30
  • 1
    this was super useful thanks a ton probably should have been an edit on Elizabeth's answer but I'm very happy, unix.stackexchange.com/q/694795/3285 Mar 17, 2022 at 14:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .