Terminating Endpoints and Draining
When a rolling update deletes a backing Pod, why do requests keep flowing instead of failing? Here the replacement is Ready first, and on deletion the old endpoint turns ready=false and terminating=true, so kube-proxy rewrites the Service rules to send new connections to the ready Pods. A preStop sleep gives that rewrite time to land before SIGTERM, established TCP flows finish on their conntrack entries, and the Pod exits inside its grace period.
Play the animation 6 steps, animated, best on a wide screen
Terminating Endpoints and Draining, step by step
-
The slice for the Service web lists endpoints for web-a and web-c, drawn here for web-c: ready=true, serving=true, terminating=false. A long request lands on web-c and a new connection on web-a, each pinned to its Pod by a conntrack entry in the Node dataplane.
Play from step 1 -
With two replicas the default maxUnavailable of 25% rounds down to 0 and the default maxSurge of 25% rounds up to 1, so the rollout adds web-d before it removes anything. Once its readiness probe passes, its endpoint turns ready=true and the Service rules send it new connections too.
Play from step 2 -
Only then does the rollout delete web-c, and the grace clock starts at 0s. At once the 5s preStop sleep web-c declares begins, and its endpoint turns ready=false and terminating=true, while serving follows readiness and stays true.
Play from step 3 -
Inside that 5s window kube-proxy reads the flip and, as ready endpoints remain, rewrites the Service rules without web-c, so a new connection lands on web-a. The sleep exists because this update races the shutdown: web-c must keep answering until it lands.
Play from step 4 -
SIGTERM arrives at 5s as the sleep ends, and Pod web-c takes until 14s to finish what it holds. The long request still reaches it on its established TCP conntrack entry, which the rule change does not touch, while the next new connection goes to web-d.
Play from step 5 -
Pod web-c exits at 14s, well inside the 30s grace period, so SIGKILL is never sent. Once the Pod reaches a terminal phase, its endpoint leaves the slice and the Pod object is removed. Pods web-a and web-d carry the Service on, and no request was dropped.
Play from step 6