Environment Before PID 1

How does a ConfigMap value reach a container before its process starts? At launch the Kubelet resolves env, envFrom and the downward API into one set of variables and hands it over in CreateContainer, adding variables per Service with a cluster IP in the namespace at that instant unless enableServiceLinks is false. That set is a copy taken once, so a later edit moves the object, never the running process, and only a restarted container reads the new value.

Play the animation 5 steps, animated, best on a wide screen

Environment Before PID 1, Kubernetes diagram

Environment Before PID 1, step by step

  1. One container gets three variables and each comes from a different kind of place. DB_HOST is a key in a ConfigMap that somebody else owns and edits, MY_POD_IP names a field of the Pod itself, status.podIP, through the downward API, and WEB_SERVICE_HOST is never asked for at all: by default the Kubelet adds Service variables on its own. None of the three is resolved yet.

    Play from step 1
  2. The Kubelet reads the objects the spec named, and it reads them at the moment it launches the container rather than at the moment the Pod was created. DB_HOST takes whatever the key holds at that instant, and a Secret is opened the same way. The Pod IP costs no read at all, because the Kubelet set up the Pod sandbox and already knows its address.

    Play from step 2
  3. On top of what the spec asked for, and unless enableServiceLinks is false, the Kubelet writes variables for every Service in this namespace that has a cluster IP right now, among them WEB_SERVICE_HOST and WEB_SERVICE_PORT for a Service named web. That list is a snapshot, so a Service created one second later is absent from this container. DNS is the way out of that ordering trap.

    Play from step 3
  4. The whole set crosses in one call. CreateContainer carries it, PID 1 starts with the variables already in its environment, and no second call of that kind is ever made for this container. What the downward API can send across has a limit worth knowing: a named label or annotation comes through, but the whole metadata.labels map is available only as a file in a downwardAPI volume.

    Play from step 4
  5. Somebody edits the ConfigMap. The Kubelet sees it, because by default it watches, and the object above the line now says something the copy below the line does not. A mounted volume without subPath would pick the edit up on a later sync and a downwardAPI volume does the same after a resize, but a variable is a copy taken once, and only a restarted container reads it again.

    Play from step 5

Sources