Container Termination Order
Which container in a multi-container Pod is stopped first, and why is it not the sidecar? Regular containers are signalled first, at different times and in no guaranteed order, and only once the last of them has terminated does the Kubelet signal the native sidecars, in the reverse of the order they are declared in. That keeps a proxy or a log agent alive while the app drains, and every stop is spent from one shared terminationGracePeriodSeconds.
Play the animation 6 steps, animated, best on a wide screen
Container Termination Order, step by step
-
The Pod is marked for deletion and the Kubelet begins the local shutdown on this Node. Nothing has been signalled yet. Five containers are running here, three declared in spec.initContainers with restartPolicy Always and two in spec.containers, and one terminationGracePeriodSeconds covers all of them.
Play from step 1 -
Every container in spec.containers goes first. The Kubelet runs any preStop hook they declare and has the runtime send the stop signal to web and to redis, at different times and in an arbitrary order, so which of the two goes first is not something to rely on. No sidecar above has been signalled yet.
Play from step 2 -
That gate is the reason the order runs this way. While web and redis are still winding down, mesh-proxy is still carrying their outbound calls, log-agent is still tailing the file they write and otel-agent is still shipping what they emit. A sidecar stopped alongside the app would cut a path the app still needs.
Play from step 3 -
The last container in spec.containers has fully terminated, so the gate opens and the sidecars follow, in the reverse of the order they are declared in. The row is walked from its end: initContainers[2] is otel-agent, so otel-agent is the first sidecar to be signalled.
Play from step 4 -
Next is initContainers[1], which is log-agent, while mesh-proxy at initContainers[0] is still running. That is what lets log-agent flush its last lines out through mesh-proxy on the way down, and it is the whole reason the row is walked backwards rather than forwards.
Play from step 5 -
The mesh-proxy container is declared first, so it is stopped last, which is what kept a route open for the four that went before it. All five stops came out of the one budget the Pod carries, so if it runs out first, every sidecar not yet signalled is stopped at once with a short grace period.
Play from step 6