CSI Storage Capacity
Why does a Pod get stuck Pending when the local storage pool on its chosen Node is full? Without capacity information the Scheduler picks that Node with no view of its free space, provisioning fails there, and the Node choice is reset so the Pod is placed blind again on the next pass. CSIStorageCapacity objects from a driver that opts in let the Scheduler see free space per topology segment and filter first, on classes that bind WaitForFirstConsumer.
Play the animation 5 steps, animated, best on a wide screen
CSI Storage Capacity, step by step
-
Without capacity tracking the Scheduler judges Nodes on CPU, memory, affinity and the rest, but not on free storage, and Node-1 wins. It records Node-1 as the selected Node on the claim and waits for a volume there before it binds the Pod. Only 5Gi is free in the Node-1 pool, but nothing tells the Scheduler that.
Play from step 1 -
The external-provisioner calls CreateVolume for 20Gi on Node-1, and the driver answers ResourceExhausted because that pool has only 5Gi free. The driver reports topology, so the external-provisioner drops the selected Node and scheduling starts over. With the same inputs it can land on Node-1 again, and the Pod stays Pending.
Play from step 2 -
Turn on capacity tracking. The external-provisioner, started with --enable-capacity, calls GetCapacity for each topology segment and publishes one CSIStorageCapacity object per StorageClass and segment, by default only for WaitForFirstConsumer classes. A segment here is one Node: 5Gi for Node-1, 50Gi for Node-2.
Play from step 3 -
The Scheduler uses those objects because storageCapacity is true on the CSIDriver, the class binds WaitForFirstConsumer and the claim still needs a new volume. Its filter phase compares the 20Gi claim with maximumVolumeSize when the driver reports it, else with capacity. 20Gi does not fit in 5Gi, so Node-1 is filtered out before scoring and Node-2 is the only candidate.
Play from step 4 -
The Scheduler records Node-2 as the selected Node, the CSI controller creates the volume in the Node-2 pool, the claim binds, and only then is the Pod bound to Node-2 and started. It finds room on the first try because the Scheduler looked before it picked, though capacity that changes after it was published can still force a retry.
Play from step 5