4

I'm using lxc to have a ubuntu 16.04 development environment on my 18.04 laptop. When I do a parallel build (ninja -j) on the container my computer becomes unresponsive and never recovers. I have to restart it when this happens. I know this is vague, I am suspicious this has something to do with memory usage or some other resource that when building on the host is better managed. If I remember to do -j 4 (gnu compilers) it does not lock up.

I've setup lxc in the simplest way where I have to run it as root. It does not have access to network devices and I "mount" folders on it using the config file to share repos with it. Below is my config file:

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf

# For Ubuntu 16.04
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0
lxc.arch = linux64

# Container specific configuration
lxc.rootfs.path = dir:/var/lib/lxc/u2/rootfs
lxc.uts.name = u2

# Network configuration
lxc.net.0.type = none
lxc.net.0.flags = down

# Share Display for gui applications
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir
lxc.mount.entry = /dev/video0 dev/video0 none bind,optional,create=file

# Share folders
lxc.mount.entry = /home/tyler/workspace /var/lib/lxc/u2/rootfs/home/ubuntu/workspace none bind 0 0

My question has two parts, how can I isolate what is causing the lockup, and how can I configure my LXC container to keep it from locking up my computer when doing parallel builds?

  • Does your host system have a swap file? – 111--- Oct 4 '19 at 15:14
  • Yes, it is only 1GB, I'll try increasing it and see if that helps. – tylerjw Oct 4 '19 at 21:59
  • I added an additional 8gb of swap (total of 9gb) and that seems to have made it work. Watching the swap usage on htop makes me wonder why that would have worked because I never see it using more than 300M of swap. – tylerjw Oct 4 '19 at 22:34
  • 1
    After more use I do think this was the problem. My computer is now setup with 9GB of swap and htop regularly shows it using ~3GB. Before I only had 1GB. Thank you for your help @111 – tylerjw Oct 17 '19 at 18:35
2
+50

It sounds like your LXC host is running out of memory and the system is killing processes. You have a couple of options:

  1. Add more memory or add a swap file / increase swap to your host
  2. Limit your LXC dev container to one or more CPU cores to make parallel ninja builds less aggressive

For option 2, assuming you have a 4 core CPU, following LXC commands should limit your container to 2 cores and give it scheduling time on 50% of those cores (effectively a 75% reduction in CPU access).

lxc config set container1 limits.cpu 2
lxc config set container1 limits.cpu.allowance 50%

(container1 above is your lxc dev container name)

Adjust the number of CPUs first. The 'cpu.allowance' command will probably have less of an impact on your issue if your host is running out of memory.

With fewer CPU cores available to the guest container, ninja should kick off less parallel build commands and thus use less system resources (memory specifically).

Edit

To make these changes without using LXD commands, edit the container's config file and add the following line:

lxc.cgroup.cpuset.cpus = 0-3

This will give the container cores 0 and 3, adjust to suit.

Here is some further reading on lxc cgroup config parameters.

  • the lxc config command actually commands lxd. Is there a way to do this without using lxd? – tylerjw Oct 4 '19 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.