Do docker containers share RAM for files memory mapped from the same layer but a different image?

Question

I'm not 100% certain about whether this is a U&L question or a SO question. On balance I'm posting it on U&L as it's OS related.

Background

As far as I know, Linux will load shared libraries (.so files) by memory mapping them as copy-on-write. One advantage of this is that multiple processes which share the same large library will all share the same physical RAM for much of that library's content.

This doesn't necessarily happen with Docker because processes run in their own "container" based off an "image" and each image contain's it's own copy of shared libraries. This is deliberate. It allows programs to ship with their own dependencies (libraries) which may be substantially different to the libraries already installed on the system.

So a program running natively on a docker host will not share the same memory for libraries as a program running inside a docker container because the program in the docker container has mapped to different copies of the libraries.

Docker layers explained

Docker images are created in layers. Each layer adds to the lower one, sometimes overwriting existing files. Not every file is changed in every layer.

Docker allows you create new images by adding new layers to an older image. When this happens you end up with multiple images sharing the same layers. The images share identical copies of some of the same files.

Docker keeps the layers separately, at least before runtime. Eg: wen pulling an image from Docker Hub, Docker fetches images by fetching each image's constituent layers. It only fetching the layers it doesn't already have.

What I don't know

When creating or running a container Docker must assemble the layers into a single coherent file system. I don't know how it does this. It could:

• Copy the files into one place
• Create hard links into one place
• Use an overlay file system

Depending on what it does, files which originated from the same layer may be identical copies, or they may the exact same file on the file system.

This will ultimately affect what happens when the files are memory mapped by multiple processes.

What am I really trying to discover?

I want to know if running two containers from two different images will share the same RAM for a single shared library that originated in a single layer.

Answer 1

At least in some configurations, yes, containers can share memory mappings for files in the same layer in different images.

Here’s an experiment to demonstrate this. I’m using two different images, one based on the other:

$ docker history 5f35156022ae
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
5f35156022ae   7 weeks ago    COPY scripts/shared/ . # buildkit               1.05MB    buildkit.dockerfile.v0
<missing>      7 weeks ago    WORKDIR /opt/shipyard/scripts                   0B        buildkit.dockerfile.v0
...

$ docker history 569bf4207a08
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
569bf4207a08   7 weeks ago    /bin/sh -c #(nop)  CMD ["sh"]                   0B        
ed9510deb54e   7 weeks ago    /bin/sh -c #(nop)  ENTRYPOINT ["/opt/shipyar…   0B        
c3e0351f0dd2   7 weeks ago    /bin/sh -c #(nop) WORKDIR /go/src/github.com…   0B        
a476f9f2b118   7 weeks ago    /bin/sh -c #(nop)  ENV DAPPER_OUTPUT=/go/src…   0B        
29a76c4ff3e7   7 weeks ago    /bin/sh -c #(nop)  ENV DAPPER_ENV=QUAY_USERN…   0B        
2f4a590d61ef   7 weeks ago    /bin/sh -c #(nop)  ARG PROJECT                  0B        
5f35156022ae   7 weeks ago    COPY scripts/shared/ . # buildkit               1.05MB    buildkit.dockerfile.v0
<missing>      7 weeks ago    WORKDIR /opt/shipyard/scripts                   0B        buildkit.dockerfile.v0
...

I started two containers, just using the entrypoint shell:

$ pstree -p
...
           ├─containerd-shim(530457)─┬─bash(530477)
           │                         ├─{containerd-shim}(530458)
...
           ├─containerd-shim(530622)─┬─entry(530643)───sh(530685)
           │                         ├─{containerd-shim}(530624)
...

Let’s examine the C library used by both these shells:

$ sudo grep libc-2.33 /proc/{530477,530685}/maps
/proc/530477/maps:7fc127f81000-7fc127fa7000 r--p 00000000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530477/maps:7fc127fa7000-7fc1280f4000 r-xp 00026000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530477/maps:7fc1280f4000-7fc128140000 r--p 00173000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530477/maps:7fc128140000-7fc128141000 ---p 001bf000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530477/maps:7fc128141000-7fc128144000 r--p 001bf000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530477/maps:7fc128144000-7fc128147000 rw-p 001c2000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5df94000-7f6a5dfba000 r--p 00000000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5dfba000-7f6a5e107000 r-xp 00026000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5e107000-7f6a5e153000 r--p 00173000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5e153000-7f6a5e154000 ---p 001bf000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5e154000-7f6a5e157000 r--p 001bf000 00:1f 3117                       /usr/lib64/libc-2.33.so
/proc/530685/maps:7f6a5e157000-7f6a5e15a000 rw-p 001c2000 00:1f 3117                       /usr/lib64/libc-2.33.so

Both mapped libraries have the same device and inode, so they’re the same file, and their mappings will be shared where possible.