Cilium Architecture: CCA practice questions

Cilium's architecture is easiest to understand as a control plane running in user space and a datapath running in the kernel. The agent on each node receives configuration from Kubernetes, compiles and loads eBPF programs, and fills the eBPF maps those programs read. Once loaded, the programs do the per-packet work: identity lookup, policy, load balancing and forwarding.

The datapath is built from kernel hooks. XDP runs earliest, in the network driver, and Cilium uses it for a prefilter that drops traffic based on CIDR maps. Traffic control (tc) hooks run after initial stack processing; each pod is connected to the host with a veth pair, and the program on the tc ingress hook of the host-side veth sees every packet that leaves the container. Socket-level programs translate Service addresses when an application connects and speed up redirects between local sockets. Together with the virtual interfaces cilium_host and cilium_net, an optional overlay device and the Envoy proxy, these hooks form logical objects. The Endpoint Policy object maps a packet to an identity and applies L3 and L4 rules, then drops it, delivers it locally, passes it to the Service object or sends it to the L7 Policy object.

Because the programs and maps are pinned in the BPF filesystem at /sys/fs/bpf, they survive a restart of the agent. Forwarding continues while the agent is upgraded, and Cilium mounts the filesystem itself if the host has not. The agent also probes the running kernel and uses newer features when they are present; the current minimum is Linux 5.10, or 4.18 on RHEL 8.10.

State that agents must share, such as identities, is stored in Kubernetes custom resources by default. An etcd key-value store is optional and is described as a scalability optimization for large clusters, because change notifications are more efficient there.

The model of the cluster is expressed in a few objects. An endpoint is the group of containers that share one IP address, normally a pod, with an ID unique on its node. For each managed pod Cilium creates a CiliumEndpoint resource with the same name and namespace that carries its labels, security identity and policy state, so kubectl get ciliumendpoints gives a cluster-wide view. Each agent also creates a cilium-health-<node> endpoint in kube-system for its own health checks, which use ICMP and TCP 4240 between nodes; if a firewall blocks both, Cilium keeps working but reports no health data.

The agent pod runs in the host network namespace and needs CAP_SYS_ADMIN to install eBPF programs system-wide; the standard DaemonSet grants this automatically.