Container Filesystem Layers
Why does a file written in a container vanish when it is replaced, while one under a volume stays? With containerd and its default overlayfs snapshotter the root filesystem is one overlay mount: a file read takes the highest layer holding the path, and every change to that mount lands in the writable upperdir as a new file, a copied-up file or a whiteout. Image layers never change, the upperdir goes when the container is removed, and a volume at /data skips it all.
Play the animation 6 steps, animated, best on a wide screen
Container Filesystem Layers, step by step
-
The top row is what the container sees: one overlay mount over the layers below it. It reads /etc/app.conf, and a file lookup takes the highest layer holding that path. The upperdir and the app layer have none, so the base layer serves it.
Play from step 1 -
The container creates /tmp/cache. No image layer can be written, so the new file lands in the upperdir, the one writable layer this container owns, and the merged view shows it from there.
Play from step 2 -
Now the container edits /etc/app.conf. Overlayfs first copies the whole file up from the base layer into the upperdir, and the edit lands on that copy. The merged view now resolves to the upper copy, and the base copy is hidden but unchanged.
Play from step 3 -
Deleting /bin/tool cannot touch the app layer, which is read-only. Overlayfs writes a whiteout for that name into the upperdir instead, and the merged view no longer shows the file. The app layer still holds it.
Play from step 4 -
The container writes db under /data, and that write never enters the overlay. /data is a volume, a separate mount placed over the merged tree at that path, so the bytes skip every layer row and land on the volume.
Play from step 5 -
The container is replaced, and the merged row now shows the new one. It starts with a new, empty upperdir, so the cache, the edited app.conf and the whiteout are gone, /bin/tool shows again, and app.conf comes from the base layer. The old upperdir is deleted once the old container is removed. Only the volume still holds db.
Play from step 6