Architecture: CCA practice questions

Cilium is built from a small set of cooperating programs, and most architecture topics come down to knowing which program owns which job. The Cilium agent, cilium-agent, runs on every node of the cluster (it is deployed as the cilium DaemonSet). It takes configuration from Kubernetes or its own API and turns it into networking, load balancing and network policy, and it manages the eBPF programs that the Linux kernel uses to control all traffic in and out of the containers on that node.

The Cilium operator handles tasks that should be done once for the whole cluster rather than once per node. The documentation is explicit that the operator is not in the critical path for any forwarding or network policy decision; if it is unavailable, the visible effects are delays, for example in IP address allocation and in kvstore heartbeat updates, not dropped traffic. Two command-line tools are easy to confuse. The cilium CLI is run from your workstation and talks to the cluster through kubeconfig, while cilium-dbg runs on a node, queries the REST API of the local agent and can read eBPF maps directly. The CNI plugin, cilium-cni, is invoked by Kubernetes whenever a pod is scheduled or terminated on a node, and it asks the local agent to set up or tear down that pod's datapath.

Hubble provides observability on top of the same datapath. The Hubble server is embedded in the agent on each node, Hubble Relay connects to all of those servers to give a cluster-wide view, the hubble CLI reads from Relay or from a local server, and Hubble UI draws a service dependency map from Relay data.

eBPF itself is the in-kernel engine. Programs are checked by an in-kernel verifier before they run and are translated to native code by a JIT compiler, and Cilium probes the kernel and uses newer features when they are present. State is stored in Kubernetes custom resources by default; an etcd key-value store is optional and exists as a scalability optimization.

Policy is tied to identity rather than addresses. A pod's security identity is derived from its labels, pods with the same labels share an identity, and rules allow traffic between identities, which separates security from network addressing.

For connectivity between nodes, encapsulation is the default because it asks the least of the network: nodes form a mesh of VXLAN (UDP 8472) or Geneve (UDP 6081) tunnels. Native routing, set with routing-mode: native and ipv4-native-routing-cidr, instead requires the underlying network to route pod CIDRs, either through a router or by installing routes on every node. Pod addresses come from an IPAM mode; cluster scope is the default, and others include Kubernetes host scope, multi-pool, CRD-backed and cloud modes such as AWS ENI, Azure and GKE.