I am running a dockerised jupyter notebook on a Centos server. One of the jupyter's task takes a lot of resources to complete and I noticed that my server runs slowly at that moment.

This is how I run my container, notice the shm-size argument (used by jupyter while doing parallel jobs):

docker run -t --shm-size=512m -v /opt/notebooks:/opt/notebooks -p 8888:8888 continuumio/anaconda3:ml-ready /bin/bash -c "/opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip='*' --port=8888 --no-browser --allow-root"

What I don't understand is that I can see several pythoninterpreter instances when through the top command. But none through pgrep -af python. When I stop my container, CPU's load comes back to normal and there's no python processes in the top command. So it looks like Docker processes are visible on the host, as far as I know it is impossible, so what is happening ?

  • pgrep -af searches through the command lines of the processes, which doesn't include python in your case. top shows the executed binary, which happens to be python. Try pressing c in top to show command lines instead. – muru Feb 19 '18 at 10:33
  • I guess I am just surprised that commands launched within docker appears on the host. I should reformat my post or do another one. Thanks anyway for the useful c trick, really neat. – Totem Feb 19 '18 at 15:10
  • 1
    It's normal for the host to see containerised processes. The containers themselves are limited in what they can do or see, not the host. BTW, finding python processes with ps can sometimes be problematic because the python community believes that running python scripts with #!/usr/bin/env python is a good idea. This mangles ARGV[0], which makes it difficult to find specific named scripts with ps alone. – cas Feb 20 '18 at 5:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.