Where is the actual executable program code located in the Docker architecture?

Question

I am new to using Docker, so this may seem like a silly question, but I just can't figure it out. I already learned that docker images contain the zipped libraries and dependencies needed to run the application, and that containers are created from these images.

However, what I don't understand is how this zipped code in the image is finally executed inside the container. In previous research I found that the containers contain copies of this code in their filesystem and that they add a write layer to this code in order to be runnable. That is why I find it strange to see that in practice images take much more storage than the entire filesystem of the containers based on them. You would expect that since the containers have the unzipped code in their filesystem, these containers would be much larger than their images.

Probably, I am completely misunderstanding how docker works, but can someone please help me? Thanks in advance!

Answer 1

A docker image is a tar file containing tar files!

For example, if we have a very simple docker container

% cat Dockerfile
FROM scratch
COPY hello /
CMD ["/hello"]

And if we build this with docker build -t hello . then the resulting image can be pulled apart:

% mkdir /tmp/container
% cd /tmp/container
% docker save hello | tar xvf -
1cb4c0f3e212f5abfb4b0b74d36a6ace8b7fad164b3e460d8cbf7fb1c3905270.json
59aef640ce511eadb6169072a6d8cefd95a3d1b14b73b92bd86fcb7e0b67618e/
59aef640ce511eadb6169072a6d8cefd95a3d1b14b73b92bd86fcb7e0b67618e/VERSION
59aef640ce511eadb6169072a6d8cefd95a3d1b14b73b92bd86fcb7e0b67618e/json
59aef640ce511eadb6169072a6d8cefd95a3d1b14b73b92bd86fcb7e0b67618e/layer.tar
manifest.json
tar: manifest.json: implausibly old time stamp 1970-01-01 00:00:00
repositories
tar: repositories: implausibly old time stamp 1970-01-01 00:00:00

The content is in the layer file

% tar tf 59aef640ce511eadb6169072a6d8cefd95a3d1b14b73b92bd86fcb7e0b67618e/layer.tar
hello

If we have multiple COPY commands then we see multiple layers

% cat Dockerfile
FROM scratch
COPY hello /
COPY lib64/ /lib64/
CMD ["/hello"]

and the result:

% docker save hello-twolayers | tar xvf -
10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/
10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/VERSION
10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/json
10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/layer.tar
1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/
1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/VERSION
1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/json
1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/layer.tar
6d706ccc02fc6c6ead9b6913b8bd21f15a34abcb09b5deb8e5407af5f4203f14.json
manifest.json
tar: manifest.json: implausibly old time stamp 1970-01-01 00:00:00
repositories
tar: repositories: implausibly old time stamp 1970-01-01 00:00:00

Each COPY command created a new "layer"

% tar tf 10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/layer.tar
hello

% tar tf 1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/layer.tar
lib64/
lib64/ld-linux-x86-64.so.2
lib64/libc.so.6

The manifest file describes the layer order

% jq '.[] | [ .Layers ]' < manifest.json
[
  [
    "10e6ef63539659eef459b5b0795ad1d91e44b4d3d6b5cf369f026cabbef333f4/layer.tar",
    "1f5263a466b60441b1569ae0bf6d059f6a866d721bde8c8e795313116400fb4f/layer.tar"
  ]
]

If we take a more normal container, say one based on Debian:

% cat Dockerfile
FROM debian:latest
COPY hello /
CMD ["/hello"]

Now if we look at the contents we can see two layers, and one of them has 8259 files in it!

% docker save hello-debian | tar xf -
tar: manifest.json: implausibly old time stamp 1970-01-01 00:00:00
tar: repositories: implausibly old time stamp 1970-01-01 00:00:00

% for a in */layer.tar; do print -n "$a: " ; tar tf $a | wc -l ; done
7a83b1430fe3eaed26ecd9b011dcd885c6f04e45bee3205734fea8ce7382a01e/layer.tar: 1
a1ae6cd0978a7d90aea036f11679f31e1a3142b1f40030fb80d4c8748b0a6a01/layer.tar: 8259

(These examples taken from https://www.sweharris.org/post/2017-06-18-buildcontainer/ so the debian image is 6 years old, but the concepts are the same).

Now notice each layer has a long number like "a1ae6cd0978a7d90aea036f11679f31e1a3142b1f40030fb80d4c8748b0a6a01". This is the sha256 hash of that layer. If two different containers were based off the same debian image then they would have this consistent layer, and so the docker daemon only stores it at the backend once.

And once the container is executed the docker daemon typically overlays (readonly) these layers so they don't use any extra disk space.

So, for example, I can see

% grep docker /proc/mounts | head -1
overlay /var/lib/docker/overlay2/c520fff24fdac7d341e240f62e64a8301f8da9c0d44c5bb5fc62a419efdf6a67/merged overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/QVO5L3KITI4IVHIF3CSS7A55ZP:/var/lib/docker/overlay2/l/CB5FLU2Q5C4WAHSSNNA7IA2LOB:/var/lib/docker/overlay2/l/K2DP5JPIRNFNMEREPMNX7QOHO7:/var/lib/docker/overlay2/l/APTRCOCNEYQROETHW7VZJ7AVIJ:/var/lib/docker/overlay2/l/K53M4VNLYJ3VWSMVOBS67TCUYZ:/var/lib/docker/overlay2/l/DKUICFXQR7PSTEBCBMXBFUNTIX:/var/lib/docker/overlay2/l/YQZ5RLIO5FR6W5MVDDMZNWZRRS:/var/lib/docker/overlay2/l/OPZULDPCI2FQPL4H2ENNKBUB3C:/var/lib/docker/overlay2/l/SHZQ6FNTIP3O5T2NUTHL3CR2GA:/var/lib/docker/overlay2/l/ZOAGXZVNGH2DY76LIBZUPN3NWZ,upperdir=/var/lib/docker/overlay2/c520fff24fdac7d341e240f62e64a8301f8da9c0d44c5bb5fc62a419efdf6a67/diff,workdir=/var/lib/docker/overlay2/c520fff24fdac7d341e240f62e64a8301f8da9c0d44c5bb5fc62a419efdf6a67/work 0 0

We can see /var/lib/docker/overlay2/c520fff24fdac7d341e240f62e64a8301f8da9c0d44c5bb5fc62a419efdf6a67/merged is the directory for the container and all the layers that go together to make it work.