Could some explain what the command apt-get update does and when I really should use it?
apt-get update downloads updated indexes from the distribution's package repositories, listing all available packages and their precise versions.
Common distributions like Ubuntu and Debian are usually conservative and backward compatible in their package offerings, so the versions won't change very much over time; they will change due to security updates or bug fixes. For example, mysql could be upgraded from
5.7.19 but not to
Where is stored the package index? On a database? On a file?
It's usually stored in one or more files inside
/var/lib/apt. In the context of Docker, these files are inside the image. When building the Dockerfile, they get stored in the new layers of filesystem that get created and persisted as the newly built image.
What happens if I do apt-get install without updating the cache?
You may try to download package versions that do not exist anymore. This is quite common on virtual machines, but it is possible inside containers too if the distribution repositories have released new packages after the base image was built. There may be no coordination between the distribution maintainers and the Dockerfile maintainers, which are downstream from the distribution and may be larger in number. There is only one Debian repository but thousands of
jessie-based container images and Dockerfile.
Moreover, some upstream images like the ubuntu one remove the downloaded index to make the image smaller and avoid outdated files there. So it's expected an updated index should be downloaded when building on top of a base image, not for each version of a base image to ship with the latest index.
Is there a chance that the remote package would not exist anymore and that the link would be broken?
Definitely, because the versions stored in the index are very precise like
5.7.19 (simplification; they are more similar to
Is there some agreed politic about deb repositories? For example, should a repository only contains the last version of a package, or on the contrary should it contains all versions available for a specific distribution release?
It's common for old minor versions to be removed quickly once an update is available; I assume this is to save space on the servers as binaries can weigh several tens of megabytes, multiplied by all the versions and architectures supported. So it is usually impossible to pin, say,
mysql-5.7.18 in the subsequent
apt-get install; as soon as
mysql-5.7.19 is released in the distribution the previous one will get removed.
To be fair to Docker, this non-determinism of
apt-get update is an issue that is brought over as part of the package management of each distribution. You would have the same problem trying to build repeatably an EC2 or Vagrant virtual machine.
Some system administrators use services such as Aptly to mirror the original repositories and be able to pin a particular version, but you run the risk of missing out on security updates unless you have a frequently-run separate process for testing the updates and changing what you are pinning.