Pod Replacement Guarantees

What actually comes back when a Pod that a controller owns disappears? A ReplicaSet and a Job both create a replacement with a new name, the ReplicaSet at once and the Job after a back-off delay, counting the loss against backoffLimit. A StatefulSet returns the identical ordinal with the claim it already had, a DaemonSet a fresh name on the Node it lost one on, and a CronJob nothing of its own, because it owns Jobs rather than Pods and leaves the Pod to its Job.

Play the animation 4 steps, animated, best on a wide screen

Pod Replacement Guarantees, Kubernetes diagram

Pod Replacement Guarantees, step by step

  1. The same thing happens to all five: the Pod is gone, and the object leaves the API. It happens at one moment and is equally true of every column, so nothing that follows is a matter of who noticed first. From here the answers are not the same, and what separates them is what each controller promises rather than how fast it reacts.

    Play from step 1
  2. Three of them hand back a NEW Pod. A ReplicaSet replica is interchangeable, so its replacement comes at once with a fresh name. A Job makes one after a back-off delay while its completions are unmet, counting the loss against backoffLimit, 6 by default. The CronJob itself starts nothing before its next tick, but its Job does.

    Play from step 2
  3. Two of them hand back something the lost Pod already had. A StatefulSet returns the identical ordinal and keeps its claim, and the cluster attaches that volume to the Node the replacement lands on. A DaemonSet returns a fresh name, but only to the Node it lost one on, because its count follows the Nodes it targets and never a number you set.

    Play from step 3
  4. The last answer is nothing at all, and here only the DaemonSet gives it. Take the Node away and its Pod is garbage collected with it, and nothing is rescheduled elsewhere, because no other Node is short of one.

    Play from step 4

Sources