LoadBalancer without a Cloud
Who gives a LoadBalancer Service its address when there is no cloud? MetalLB does it in-cluster: its controller allocates an address from a pool the operator declares, and its speakers then have to announce it to the router before any client traffic can arrive. In layer 2 mode one Node answers ARP and gratuitous ARPs move the address when it fails, while in BGP mode every Node advertises it and the router spreads connections by ECMP.
Play the animation 4 steps, animated, best on a wide screen
LoadBalancer without a Cloud, step by step
-
On bare metal no cloud provider implements load balancers, so a Service of type LoadBalancer stays pending. MetalLB fills that gap in-cluster. Its controller takes a free address from the IPAddressPool the operator declared, 203.0.113.0/24, and writes 203.0.113.9 into status.loadBalancer.ingress. An allocated address is not a reachable one: no router knows where it lives yet.
Play from step 1 -
In layer 2 mode every speaker sorts the eligible Nodes by a hash of Node and address, and the first Node owns 203.0.113.9. Only the speaker on Node-1 answers ARP for it, so the router sends every packet for the address to Node-1. From there kube-proxy, under the default Cluster policy, spreads connections across all ready Pods of the Service, this one to the local Pod. No router configuration is needed.
Play from step 2 -
Node-1 fails. The other speakers detect it through memberlist and drop it from the candidates, so Node-2 comes first now and its speaker sends gratuitous ARPs for the router to rewrite its entry. Failover usually takes a few seconds, and connections that went through Node-1 are lost. Even with every Node healthy, one owner limits the ingress bandwidth of the Service to that of a single Node.
Play from step 3 -
BGP mode changes the shape. The speaker on every Node peers with the router and advertises 203.0.113.9, so a router with multipath on installs three equal-cost next hops for it and typically hashes each connection onto one Node. Ingress spreads across Nodes, but router hashes are usually not stable: when a Node goes, expect all active connections to break. Resilient ECMP helps, and the router has to speak BGP.
Play from step 4