Cilium CNI

Platform9 Managed Kubernetes (PMK) supports Cilium as the Container Network Interface (CNI) for secure, high-performance cluster networking powered by eBPF. This page explains the architecture, prerequisites, installation options, configuration choices, migration guidance from Calico, and validation/troubleshooting steps aligned with PMK best practices.

For a conceptual introduction to Cilium and Hubble, see the Cilium introduction

Support Matrix and Recommendations

Best practices

• Use eBPF masquerade with kube-proxy replacement for lower latency and reduced conntrack pressure.
• Set MTU to match end-to-end path; if using VXLAN over jumbo-capable underlay, set 9000 on NICs and size Cilium MTU accordingly (account for overlay overhead).
• Ensure nodes have required kernel features for eBPF (modern kernels on Ubuntu 22.04/24.04 meet this by default).

Prerequisites

In addition to having a Kubernetes cluster provisioned and healthy and administrative kubectl access, there are requirements for using Cilium as the CNI implementation:

• Subnets on a network should not coincide. You cannot have two clusters with container CIDR or service CIDR being the same on one L2 domain or network.
• Outbound connectivity to pull Cilium images and Helm charts (or a mirrored registry).

Create a Cilium-enabled Cluster

While creating a new Kubernetes cluster using the PMK UI, in the wizard under Network Configuration, select Cilium as the network backend.

Alternative Manual Installation Options

For manual Cilium installation, choose one path: Helm (flexible, GitOps-friendly) or Cilium CLI (simplified). Ensure any legacy CNI artifacts are removed before applying Cilium.

Option A: Installation with Helm

1. Add chart repo and update indices.
2. Install cilium into kube-system with PMK-aligned values: VXLAN routing, eBPF masquerade, kube-proxy replacement, MTU sizing.
3. Verify daemonset rollout and agent health across all nodes.

Option B: Install with Cilium CLI

1. Install cilium CLI on an admin workstation.
2. Apply installation with flags that match PMK recommendations (VXLAN, eBPF masquerade, kube-proxy replacement).
3. Run post-install validation (cilium status, cilium connectivity test).

If the init container fails to copy binaries into /opt/cni/bin, fix permissions on each node:

xxxxxxxxxx

sudo chown root:root /opt/cni/bin

sudo chmod 755 /opt/cni/bin

sudo chown -R root:root /opt/cni/bin

sudo chmod -R 755 /opt/cni/bin

Post-Install Validation

1. Agent Health: Cilium status should report OK for all subsystems and nodes.

xxxxxxxxxx

kubectl -n kube-system exec ds_cilium -- cilium status

2. Connectivity: Validate pod-to-pod, pod-to-service, DNS, and egress.

kubectl run test-a --image=busybox --restart=Never -- sleep 3600

kubectl run test-b --image=busybox --restart=Never -- sleep 3600

kubectl exec test-a -- ping -c3 $(kubectl get pod test-b -o jsonpath='{.status.podIP}')

kubectl create deploy web --image=nginx --replicas=2

kubectl expose deploy web --port=80

kubectl exec test-a -- wget -qO- http://web

kubectl run -it --rm --restart=Never --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 dns -- nslookup kubernetes.default

kubectl exec test-a -- wget -qO- http://example.com

3. NetworkPolicy: Confirm deny-by-default and allow-specific behavior.

xxxxxxxxxx

apiVersion: networking.k8s.io/v1

kind: NetworkPolicy

metadata:

name: demo-deny

spec:

podSelector:

    matchLabels:

        app: web

policyTypes:

- Ingress

xxxxxxxxxx

apiVersion: networking.k8s.io/v1

kind: NetworkPolicy

metadata:

name: allow-from-test-a

spec:

podSelector:

    matchLabels:

        app: web

ingress:

- from:

    - podSelector:

        matchLabels:

            run: test-a

xxxxxxxxxx

kubectl label deploy web app=web --overwrite

kubectl apply -f demo-deny.yaml

kubectl apply -f allow-from-test-a.yaml

# Should succeed

kubectl exec test-a -- wget -qO- http://web

# Should fail

kubectl exec test-b -- wget -qO- http://web || echo "blocked as expected"

Migration from Calico to Cilium

If your Kubernetes cluster previously used Calico, perform a careful, stepwise migration to avoid stale CRDs, iptables, or CNI config conflicts. Please follow the corresponding knowledge base article.