The Netfilter Path a Packet Takes
Why must the DNAT that kube-proxy programs run before routing? A packet enters at PREROUTING, where conntrack records the flow and the nat table rewrites the Service address to a backend Pod, and only then does routing run, on the rewritten address. That decision is a fork: an address the Node owns goes to INPUT, everything else to FORWARD and POSTROUTING, where MASQUERADE waits. The reply is untangled by conntrack, walking no Service rule.
Play the animation 6 steps, animated, best on a wide screen
The Netfilter Path a Packet Takes, step by step
-
A packet leaves the client Pod through its veth pair and arrives in the host network namespace, so the kernel sees it as incoming traffic. The first hook it meets is PREROUTING, before any routing decision has run, and conntrack records a flow it has never seen.
Play from step 1 -
Inside PREROUTING the nat table runs, and the KUBE-SERVICES chain kube-proxy writes in iptables mode matches the ClusterIP. It rewrites the destination to a real backend, 10.244.2.7:8080 on Node-2, and conntrack stores that translation beside the flow. What leaves this hook is no longer addressed to a Service.
Play from step 2 -
Only now does the kernel decide where to send the packet, and it decides on the rewritten address. This box is the one on the row that is not a hook: it is a fork. An address the Node itself owns goes to INPUT and is delivered locally, everything else goes on to FORWARD.
Play from step 3 -
FORWARD is where the filter table runs, and where an iptables NetworkPolicy implementation drops what is not allowed. POSTROUTING is the last hook before the wire, and MASQUERADE lives there because only after routing is the outgoing interface known. By default neither kube-proxy nor the egress rules masquerade a Pod to Pod packet, so the Pod source survives.
Play from step 4 -
The reply arrives on eth0 from 10.244.2.7. At PREROUTING conntrack matches the recorded flow and sees an established connection, so no Service rule is read on the way back. The reversal itself is a source rewrite, so it lands at POSTROUTING: the source becomes 10.96.0.20 again, the address the Pod dialed, and the veth hands the packet to the Pod.
Play from step 5 -
The fork resolves the other way when the address after the rewrite belongs to the Node itself, which is what a hostNetwork backend on this Node gives it. Routing sends the packet to INPUT and a local socket has it. POSTROUTING is never reached, so MASQUERADE can never see such a packet.
Play from step 6