0

Question

How do I manage a systemd-nspawn container as a systemd unit?

Setup

Install an image

# machinectl pull-raw --verify=no https://ftp.halifax.rwth-aachen.de/fedora/linux/releases/30/Cloud/x86_64/images/Fedora-Cloud-Base-30-1.2.x86_64.raw.xz

Discover the image name

# machinectl list-images
NAME                            TYPE RO  USAGE  CREATED                     MODIF
Fedora-Cloud-Base-30-1.2.x86_64 raw  no  891.6M Fri 2019-04-26 02:14:49 UTC Fri 2

1 images listed.

Start an interactive shell inside Fedora container

# systemd-nspawn -M Fedora-Cloud-Base-30-1.2.x86_64

Write /root/app.py python3 application that handles signals

# https://stackabuse.com/handling-unix-signals-in-python/
import signal
import os
import time
import sys

def terminateProcess(signalNumber, frame):
    print(f'received signal {signalNumber}')
    print ('exiting...')
    sys.exit()

def receiveSignal(signalNumber, frame):
    print(f'received signal {signalNumber}')
    return

if __name__ == '__main__':
    # register the signals to be caught
    signal.signal(signal.SIGHUP, receiveSignal)
    signal.signal(signal.SIGINT, terminateProcess)
    signal.signal(signal.SIGQUIT, receiveSignal)
    signal.signal(signal.SIGILL, receiveSignal)
    signal.signal(signal.SIGTRAP, receiveSignal)
    signal.signal(signal.SIGABRT, receiveSignal)
    signal.signal(signal.SIGBUS, receiveSignal)
    signal.signal(signal.SIGFPE, receiveSignal)
    #signal.signal(signal.SIGKILL, receiveSignal)
    signal.signal(signal.SIGUSR1, receiveSignal)
    signal.signal(signal.SIGSEGV, receiveSignal)
    signal.signal(signal.SIGUSR2, receiveSignal)
    signal.signal(signal.SIGPIPE, receiveSignal)
    signal.signal(signal.SIGALRM, receiveSignal)
    signal.signal(signal.SIGTERM, terminateProcess)

    # output current process id
    print(f'pid {os.getpid()}')

    # wait in an endless loop for signals 
    while True:
        time.sleep(1)

Exit container using key combination

Control + ]]]

app.service attempt 1

Write /etc/systemd/system/app.service unit file

[Service]
ExecStart=/usr/bin/systemd-nspawn --keep-unit -M Fedora-Cloud-Base-30-1.2.x86_64 python3 -u /root/app.py
SyslogIdentifier=%N
  • --keep-unit switch keeps systemd-nspawn and app.py in the system.slice/app.service cgroup
  • -u switch for unbuffered output
  • SyslogIdentifier using %N specifier for the string "app", that's the unit name without the suffix

Reload systemd daemon

# systemctl daemon-reload

In another terminal, continously follow log output with systemd-journald

# journalctl -f -u app.service

Start app.service unit

# systemctl start app.service

Stop app.service unit

# systemctl stop app.service

Observe logs

-- Logs begin at Fri 2019-08-23 16:58:11 UTC. --
Aug 23 17:26:42 srv0 systemd[1]: Started app.service.
Aug 23 17:26:42 srv0 app[12745]: Spawning container Fedora-Cloud-Base-30-1.2.x86_64 on /var/lib/machines/Fedora-Cloud-Base-30-1.2.x86_64.raw.
Aug 23 17:26:42 srv0 app[12745]: Press ^] three times within 1s to kill container.
Aug 23 17:26:42 srv0 app[12745]: Failed to create directory /tmp/nspawn-root-afZQoJ/sys/fs/selinux: Read-only file system
Aug 23 17:26:42 srv0 app[12745]: Failed to create directory /tmp/nspawn-root-afZQoJ/sys/fs/selinux: Read-only file system
Aug 23 17:26:42 srv0 app[12745]: pid 1
Aug 23 17:26:54 srv0 systemd[1]: Stopping app.service...
Aug 23 17:26:54 srv0 app[12745]: Container Fedora-Cloud-Base-30-1.2.x86_64 terminated by signal KILL.
Aug 23 17:26:54 srv0 systemd[1]: app.service: Main process exited, code=exited, status=1/FAILURE
Aug 23 17:26:54 srv0 systemd[1]: Stopped app.service.
Aug 23 17:26:54 srv0 systemd[1]: app.service: Unit entered failed state.
Aug 23 17:26:54 srv0 systemd[1]: app.service: Failed with result 'exit-code'.
Aug 23 17:26:54 srv0 systemd[1]: Stopped app.service.

systemd-nspawn is using SIGKILL instead of SIGTERM.

See the line Aug 23 17:26:54 srv0 app[12745]: Container Fedora-Cloud-Base-30-1.2.x86_64 terminated by signal KILL. I don't want to SIGKILL app.py, I want to SIGTERM it.

app.service attempt 2

Read a github issue

https://github.com/systemd/systemd/issues/7105#issuecomment-467491778

Use the -a/--as-pid2 switch

[Service]
ExecStart=/usr/bin/systemd-nspawn --keep-unit --as-pid2 -M Fedora-Cloud-Base-30-1.2.x86_64 python3 -u /root/app.py
SyslogIdentifier=%N

daemon-reload, start, stop

Observe logs

Aug 23 17:29:59 srv0 systemd[1]: Started app.service.
Aug 23 17:29:59 srv0 app[12841]: Spawning container Fedora-Cloud-Base-30-1.2.x86_64 on /var/lib/machines/Fedora-Cloud-Base-30-1.2.x86_64.raw.
Aug 23 17:29:59 srv0 app[12841]: Press ^] three times within 1s to kill container.
Aug 23 17:29:59 srv0 app[12841]: Failed to create directory /tmp/nspawn-root-jaGbcx/sys/fs/selinux: Read-only file system
Aug 23 17:29:59 srv0 app[12841]: Failed to create directory /tmp/nspawn-root-jaGbcx/sys/fs/selinux: Read-only file system
Aug 23 17:29:59 srv0 app[12841]: pid 2
Aug 23 17:30:06 srv0 systemd[1]: Stopping app.service...
Aug 23 17:30:06 srv0 app[12841]: Container Fedora-Cloud-Base-30-1.2.x86_64 terminated by signal KILL.
Aug 23 17:30:06 srv0 systemd[1]: app.service: Main process exited, code=exited, status=1/FAILURE
Aug 23 17:30:06 srv0 systemd[1]: Stopped app.service.
Aug 23 17:30:06 srv0 systemd[1]: app.service: Unit entered failed state.
Aug 23 17:30:06 srv0 systemd[1]: app.service: Failed with result 'exit-code'.

app.py is now running as pid 2! But still receiving a SIGKILL signal instead of SIGTERM.

1 Answer 1

2

Read github issue more

https://github.com/systemd/systemd/issues/7105#issuecomment-467491778

Use the --kill-signal flag

[Service]
ExecStart=/usr/bin/systemd-nspawn --keep-unit --kill-signal=SIGTERM -M Fedora-Cloud-Base-30-1.2.x86_64 python3 -u /root/app.py
SyslogIdentifier=%N

reload systemd, start app.service, and stop app.service

systemctl daemon-reload
systemctl start app.service
systemctl stop app.service

Observe logs

Aug 23 17:51:32 srv0 systemd[1]: Started app.service.
Aug 23 17:51:32 srv0 app[12994]: Spawning container Fedora-Cloud-Base-30-1.2.x86_64 on /var/lib/machines/Fedora-Cloud-Base-30-1.2.x86_64.raw.
Aug 23 17:51:32 srv0 app[12994]: Press ^] three times within 1s to kill container.
Aug 23 17:51:32 srv0 app[12994]: Failed to create directory /tmp/nspawn-root-71uVxm/sys/fs/selinux: Read-only file system
Aug 23 17:51:32 srv0 app[12994]: Failed to create directory /tmp/nspawn-root-71uVxm/sys/fs/selinux: Read-only file system
Aug 23 17:51:32 srv0 app[12994]: pid 1
Aug 23 17:51:35 srv0 app[12994]: Trying to halt container. Send SIGTERM again to trigger immediate termination.
Aug 23 17:51:35 srv0 app[12994]: received signal 15
Aug 23 17:51:35 srv0 app[12994]: exiting...
Aug 23 17:51:35 srv0 systemd[1]: Stopping app.service...
Aug 23 17:51:35 srv0 app[12994]: Container Fedora-Cloud-Base-30-1.2.x86_64 exited successfully.
Aug 23 17:51:35 srv0 systemd[1]: Stopped app.service.
Aug 23 17:51:35 srv0 systemd[1]: Stopped app.service.

See how the SIGTERM is being passed to app.py!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .