Skip to content

StaticNetwork.NetworkSetup() leaks a duplicate iptables NAT rule on every container restart in the Knative user-container path #992

Description

@GuTS805

While reviewing pkg/network/network_static.go I noticed that setNATRule() (lines 35-89) always runs iptables -t nat -A POSTROUTING -s -o -j MASQUERADE, and never checks whether an identical rule already exists before appending it. I searched the whole repository for "iptables" and this file is the only place it shows up, so there is no matching delete call anywhere else in the codebase either.

This function is called from StaticNetwork.NetworkSetup() (same file, line 103), which is only used when getNetworkType() (pkg/unikontainers/unikontainers.go, around line 1419) detects the Knative annotation io.kubernetes.cri.container-name equal to user-container.

The problem is that Kill() (pkg/unikontainers/unikontainers.go, lines 807-838) does clean up the TAP device through network.CleanupAllUruncTaps(), but it never touches iptables or removes the NAT rule that was added on setup. In Kubernetes, when a container inside a pod restarts because of a crash or a failed liveness probe, the pod's network namespace is not recreated, only the container is. So every restart calls NetworkSetup() again, which calls setNATRule() again, which appends another identical rule to the same still-alive network namespace. Since StaticIPAddr is a fixed constant, the appended rule is byte-for-byte the same every time, so nothing ever gets deduplicated or removed. Over repeated restarts this leaves an unbounded number of duplicate NAT rules sitting in that namespace's iptables nat table.

There is already a precedent for this exact pattern of bug in this project. Issue #406 documented that the network namespace persists across container restarts in Kubernetes and that TAP devices used to leak because of it, before CleanupAllUruncTaps() was added to handle that specific resource on Kill(). The NAT rule added by the same NetworkSetup() call simply has no equivalent cleanup path, so the same underlying condition applies here but was never addressed for iptables.

I checked for duplicates by searching issues for setNATRule, NAT MASQUERADE leak, iptables, and user-container restart, and found nothing describing this. The closest related issue, #587, was about a different resource and was closed as not planned because the locking it worried about turned out to already be correct, so it does not cover this.

A fix could either make setNATRule idempotent by checking for the rule with iptables -C before appending with -A, or add a matching delete function that gets called from Kill() and Delete() the same way CleanupAllUruncTaps() already is.

This is not related to VM or sandbox isolation boundaries, it is a host network hygiene issue in the CNI setup helper, so a public issue seemed appropriate rather than private disclosure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions