Service CIDR and ClusterIP Allocation
Where does a Service ClusterIP come from when nobody assigns it? One ServiceCIDR object declares the range, and the allocator in the API server reads it as two bands: it draws from the dynamic band and falls back to the static one only when that runs out, so kube-dns on 10.96.0.10 is not reserved, only unlikely to be taken. The allocation is a write, an IPAddress object named after the address, and a second ServiceCIDR grows the range with nothing restarted.
Play the animation 4 steps, animated, best on a wide screen
Service CIDR and ClusterIP Allocation, step by step
-
One ServiceCIDR object declares the whole Service range, here 10.96.0.0/16, and the allocator reads it as two bands. Automatic allocation takes the high dynamic band first and the low static band only once that runs out. The low band is for addresses picked by hand, so the two in it are unlikely to be taken rather than reserved.
Play from step 1 -
A new Service web is created with its clusterIP field left empty, so the API server hands the request to the ClusterIP allocator. It looks in the high dynamic band, which is tried first for every automatic address, and takes one that is free, here 10.96.137.42.
Play from step 2 -
The allocation is the write. The allocator creates an IPAddress object named 10.96.137.42 that points back at the Service, and only once that object exists is the clusterIP field set. Two Services cannot end up on one address, because two objects cannot share one name.
Play from step 3 -
When the range fills up, the old fix was to widen the API server service-cluster-ip-range flag and restart it, a disruptive operation. Now you add a second ServiceCIDR object, here 10.97.0.0/16, the allocator has a second range to draw from, and the Service IP space grows with nothing restarted.
Play from step 4