Pod-to-Pod Across Nodes
What happens when the destination Pod lives on a different Node? That address is off-subnet, so the route hands the frame to the CNI dataplane. An overlay plugin such as flannel wraps the frame in VXLAN over UDP to the remote Node IP, where the kernel decapsulates it and delivers it through its bridge, while a routed plugin advertises the Pod subnet of each Node so the frame travels unwrapped, trading the MTU overhead for an underlay aware of Pod IPs.
Play the animation 4 steps, animated, best on a wide screen
Pod-to-Pod Across Nodes, step by step
-
Pod A sends to 10.244.2.7 out its eth0. The frame rides the veth into the Node-1 network stack, which consults its routing table. The destination is not in the local Pod subnet, so the route hands the frame to the CNI dataplane that carries off-Node traffic instead of to a local Pod.
Play from step 1 -
In overlay mode the CNI dataplane wraps the original frame inside a VXLAN header carried over UDP to the Node-2 address. The outer IP header carries Node IPs the physical network already knows how to route, the UDP dport is 8472 for flannel, and the inner Pod IPs ride untouched. The wrapped packet crosses the underlay to Node-2.
Play from step 2 -
Node-2 receives the UDP packet on the VXLAN port and its kernel strips the outer headers. The bare inner frame, still addressed to 10.244.2.7, is routed out of the local dataplane and over the veth into Pod B, the same last hop a same-node frame takes.
Play from step 3 -
Not every CNI encapsulates. A routed plugin such as Calico with BGP advertises the Pod subnet of each Node to its BGP peers, so the packet crosses the underlay carrying its real Pod IPs with no outer headers at all. It travels Pod A to Pod B in one routed path. This drops the encapsulation cost and the MTU overhead, at the price of an underlay that has to be aware of Pod IPs.
Play from step 4