BGP and External Networking: CCA practice questions

This topic is about making a Cilium cluster reachable from the network around it. Border Gateway Protocol is how routers exchange reachability, and Cilium's BGP Control Plane lets nodes peer with routers in the data center and advertise prefixes such as pod CIDRs or Service addresses. One limit is stated plainly in the documentation: the BGP Control Plane does not program the datapath, so it must not be used to establish reachability inside the cluster. It announces routes to the outside; it does not replace the routing mode Cilium uses between its own nodes.

The feature is switched on with the Helm value bgpControlPlane.enabled=true followed by a restart of the cilium DaemonSet, or at install time with cilium install and the same value. Configuration then lives in custom resources that each have one job.

CiliumBGPClusterConfig selects nodes with a nodeSelector and defines bgpInstances. Each instance has a name and a localASN and lists peers, each with a peerASN, a peerAddress and a peerConfigRef. That reference points at a CiliumBGPPeerConfig, which holds settings several peers can share: the address families, timers (connect retry 120 seconds, hold time 90 and keepalive 30 by default), an authSecretRef for a TCP MD5 password, graceful restart and eBGP multihop. Only ipv4/unicast and ipv6/unicast are supported families; when families are omitted, both IPv4 and IPv6 unicast capabilities are announced to the peer.

Each family carries a label selector for advertisements. A CiliumBGPAdvertisement says what to announce, and its advertisementType is PodCIDR, CiliumPodIPPool, Service or Interface. A Service advertisement chooses which addresses to announce, LoadBalancerIP, ClusterIP or ExternalIP, and selects Services by labels; it can also attach BGP communities and a local preference, the latter applying only to iBGP peers. A point that explains many empty routing tables: without a matching advertisement, no prefix is advertised to the peer, even when the session is established.

The operator turns the cluster-level configuration into one CiliumBGPNodeConfig per selected node, and that resource's status holds the live state of each instance and peer: session state, established time, route counts per address family and the negotiated timers. A CiliumBGPNodeConfigOverride, named after a node, can set that node's router ID, local port, local ASN or per-peer local address when the generated values are not right.

For day-to-day checks the cilium CLI has cilium bgp peers, which lists sessions with their state, uptime and received and advertised route counts, and cilium bgp routes, which shows the available local routes and what was advertised to each peer.

BGP also relates to the datapath choice. In native routing mode the network must know how to reach every pod CIDR; when nodes are not on one shared layer 2 segment, a routing daemon such as BGP is how those routes get distributed.