Cilium: Replace iptables with eBPF-Powered Networking in Kubernetes

Kubernetes networking has a dirty secret that most tutorials skip: the default networking implementation doesn’t scale. Every time a service is created, every time a pod comes up or goes down, every time a network policy is applied, iptables rules get rewritten. On a cluster with hundreds of services and thousands of pods, that list of rules grows into the tens of thousands. Updating it is slow. Traversing it at packet-forwarding time is even slower. This has been a known problem for years, and for a long time the answer was “use fewer services” or “get bigger nodes.” Cilium is the real answer, and it’s been the CNCF-graduated solution to this problem since it went GA in Kubernetes environments. In 2026, it is the default CNI in major managed Kubernetes offerings and the standard choice for anyone who cares about performance and observability at scale.

Why iptables Doesn’t Scale in Kubernetes

iptables was designed for firewalling on individual Linux hosts. It works well for tens or hundreds of rules. It starts falling apart for Kubernetes at scale for several reasons.

First, iptables rules are a sequential list. Packet matching walks down the list until it finds a match. With 10,000 rules, every packet potentially checks thousands of conditions before finding the right DNAT or ACCEPT rule. This is O(n) lookup time that gets worse as your cluster grows.

Second, rule updates are not atomic. When kube-proxy updates iptables to reflect a new endpoint, it locks the entire iptables table and rewrites rules in a batch. On a busy cluster this locking causes measurable latency spikes. Google’s engineering team documented cases where iptables updates on large clusters took tens of seconds.

Third, iptables has no concept of Kubernetes workloads. A rule matches an IP address. When pods get rescheduled to new IPs, all the rules referencing the old IPs need to be updated. The mapping from “Kubernetes service” to “iptables rules” is not direct — kube-proxy generates and manages this translation, adding another layer of complexity and failure modes.

The alternative that was developed to address some of these issues is IPVS (IP Virtual Server), which uses hash tables for O(1) lookups. kube-proxy supports IPVS mode and it’s better than iptables for large clusters. But IPVS doesn’t solve the observability problem, and it still runs in user space as kube-proxy, adding overhead and latency on the critical path.

What Cilium Is

Cilium is a CNI (Container Network Interface) plugin for Kubernetes that replaces iptables and kube-proxy with an eBPF-native data plane. Every packet forwarding decision, every load balancing operation, every network policy enforcement happens in the Linux kernel using eBPF programs. There is no user-space proxy in the forwarding path.

Cilium was originally developed by Isovalent (now part of Cisco) and is a CNCF graduated project. It is the default CNI in:

• Google Kubernetes Engine (GKE) Dataplane V2
• Amazon EKS with the VPC CNI using eBPF mode
• Azure AKS in certain configurations
• k3s and k0s default configurations

Cilium’s core components are:

• cilium-agent: Runs as a DaemonSet on every node. Manages eBPF programs and maps, compiles and loads network policy programs.
• cilium-operator: Cluster-level coordination — IPAM, KVstore management, certificate rotation.
• Hubble: The observability layer. Uses eBPF to capture network flow events and exposes them via gRPC and a web UI. Think of it as a distributed tcpdump with Kubernetes context baked in.

How eBPF Changes Kubernetes Networking

When Cilium replaces kube-proxy, service load balancing moves entirely into the kernel. When a pod makes a connection to a ClusterIP service, an eBPF program intercepts the connection at the socket level (before a packet is even formed) and redirects it directly to a backend pod IP. There is no DNAT in the traditional sense, no iptables chain traversal, and often no packet leaving the node at all if the destination pod is local.

This “socket-level load balancing” is one of Cilium’s most significant performance advantages. Internal east-west traffic between pods on the same node never touches the network stack at the packet level — it’s handled entirely by eBPF socket redirection. Benchmarks consistently show 30-50% latency improvements for intra-node service traffic compared to iptables kube-proxy.

Network policies work similarly. Instead of generating iptables rules that say “allow traffic from IP 10.0.1.5 to IP 10.0.2.3 on port 8080,” Cilium writes eBPF programs that say “allow traffic from pods with label app=frontend to pods with label app=backend on port 8080.” The policy is evaluated at the socket level, with cryptographic identity verification using Cilium’s identity system. If a pod’s IP changes (due to rescheduling), the policy doesn’t need to be rewritten — the identity is based on labels, not IPs.

Installing Cilium in Kubernetes

Prerequisites:

• Kubernetes 1.27 or newer
• Linux kernel 5.4 or newer on all nodes (5.10+ recommended for full feature set)
• No existing CNI installed, or you must remove it first

Install the Cilium CLI:

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
curl -L --fail --remote-name-all \
  https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz
tar xzvf cilium-linux-${CLI_ARCH}.tar.gz
mv cilium /usr/local/bin/

Install Cilium in your cluster (this also disables kube-proxy if it’s running):

cilium install --version 1.16.0

Wait for Cilium to be ready:

cilium status --wait

Run the built-in connectivity test to verify the installation:

cilium connectivity test

This test deploys test pods, verifies pod-to-pod connectivity, service load balancing, DNS, and network policy enforcement. If it passes, your Cilium installation is working correctly.

Replacing kube-proxy

For maximum performance, you want to run without kube-proxy entirely. When installing on a new cluster, pass the kube-proxy replacement flag:

cilium install \
  --version 1.16.0 \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=YOUR_API_SERVER_IP \
  --set k8sServicePort=6443

On an existing cluster where kube-proxy is already running:

# First install Cilium
cilium install --version 1.16.0 --set kubeProxyReplacement=true

# Then remove kube-proxy DaemonSet (if running in the cluster)
kubectl -n kube-system delete ds kube-proxy

# Clean up iptables rules left by kube-proxy on each node
iptables-save | grep -v KUBE | iptables-restore
ip6tables-save | grep -v KUBE | ip6tables-restore

Verify kube-proxy replacement is active:

kubectl -n kube-system exec ds/cilium -- cilium-dbg status | grep KubeProxyReplacement

Network Policy with Cilium

Cilium supports standard Kubernetes NetworkPolicy resources, but it also has its own CiliumNetworkPolicy CRD with significantly more capabilities — Layer 7 awareness, FQDN-based egress policies, and DNS-based policies.

A standard NetworkPolicy that allows traffic only from the frontend to the backend on port 8080:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - port: 8080
          protocol: TCP
  policyTypes:
    - Ingress

A CiliumNetworkPolicy that restricts a pod to only accessing a specific external domain (FQDN-based egress):

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-api-egress
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: myservice
  egress:
    - toFQDNs:
        - matchName: "api.stripe.com"
        - matchName: "api.sendgrid.com"
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
    - toEntities:
        - kube-apiserver

This policy allows myservice pods to only make outbound HTTPS connections to specific Stripe and SendGrid API endpoints, plus talk to the Kubernetes API server. All other egress is blocked. This kind of policy is nearly impossible to implement with standard iptables-based network policies.

Hubble: Network Flow Observability

Hubble is Cilium’s observability component. It taps into the eBPF programs running on every node and captures metadata about every network flow — source and destination identity, protocol, port, verdict (allow/drop), and timing. This data is aggregated by Hubble server and queryable in real time.

Enable Hubble when installing Cilium:

cilium install \
  --version 1.16.0 \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true

Install the Hubble CLI:

HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
curl -L --fail --remote-name-all \
  https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-amd64.tar.gz
tar xzvf hubble-linux-amd64.tar.gz
mv hubble /usr/local/bin/

Port-forward the Hubble relay to use the CLI:

cilium hubble port-forward &
hubble status

Observe all network flows in the production namespace:

hubble observe --namespace production

Filter for dropped traffic (policy violations):

hubble observe --verdict DROPPED --namespace production

Show flows between specific services:

hubble observe \
  --from-label "app=frontend" \
  --to-label "app=backend" \
  --protocol TCP \
  --port 8080

Filter for HTTP 5xx errors across all services:

hubble observe --http-status 5[0-9][0-9]

Access the Hubble UI for a visual network topology map:

cilium hubble ui

This opens a browser tab with an interactive graph showing all services in your cluster, the connections between them, and real-time traffic flow rates. Dropped packets and policy violations show up in red. This is the kind of visibility that previously required dedicated service mesh infrastructure with sidecar proxies and significant overhead.

Cilium Service Mesh

Beyond networking and observability, Cilium 1.14+ includes service mesh capabilities without sidecars. Traditional service meshes (Istio, Linkerd) inject a proxy sidecar into every pod. This adds memory and CPU overhead, increases latency, and complicates the deployment model.

Cilium’s service mesh uses eBPF to handle mutual TLS, traffic management, and observability at the node level rather than the pod level. No sidecar injection required. Enable it with:

cilium install \
  --version 1.16.0 \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true \
  --set ingressController.enabled=true \
  --set ingressController.default=true

With the Cilium Ingress Controller enabled, you get a Gateway API-compatible ingress implementation backed by eBPF, with Hubble providing L7 observability on all traffic.

Troubleshooting with Cilium

When network connectivity issues occur in a Cilium cluster, these commands are your first stops:

Check Cilium agent status on a specific node:

kubectl -n kube-system exec ds/cilium -- cilium-dbg status

List all eBPF endpoints managed by Cilium:

kubectl -n kube-system exec ds/cilium -- cilium-dbg endpoint list

Check the policy verdict for traffic between two pods:

kubectl -n kube-system exec ds/cilium -- \
  cilium-dbg policy trace --src-endpoint 1234 --dst-endpoint 5678 --dport 8080

Check the BPF service table (replaces iptables -t nat -L in the kube-proxy world):

kubectl -n kube-system exec ds/cilium -- cilium-dbg service list

Monitor live packet events on a node (like tcpdump but with Kubernetes context):

kubectl -n kube-system exec ds/cilium -- cilium-dbg monitor --type drop

Key Takeaways

• iptables-based kube-proxy has O(n) lookup time and table-locking that cause real performance problems at scale
• Cilium replaces both the CNI plugin and kube-proxy with eBPF programs running in the Linux kernel
• Socket-level load balancing eliminates DNAT overhead for intra-node service traffic, improving east-west latency by 30-50% in typical workloads
• Network policies in Cilium are identity-based (labels), not IP-based, so they remain valid when pods reschedule
• CiliumNetworkPolicy extends standard Kubernetes NetworkPolicy with L7-aware rules, FQDN egress filtering, and DNS-based policies
• Hubble provides real-time network flow observability with Kubernetes identity context — no sidecar proxies required
• The Hubble UI gives a live visual topology of all cluster communication with per-service metrics and policy violation highlighting
• Cilium is now the default CNI in GKE Dataplane V2, making it a production-proven choice well beyond early-adopter use cases