Understand the Observability Capabilities of Hubble: CCA practice questions

Hubble is the observability layer of Cilium. It does not add probes or agents to workloads; it reuses the visibility that the eBPF datapath already has, so flows can be recorded without changing applications. The component overview splits it into four parts. The Hubble server runs on each node inside the Cilium agent, which keeps overhead low, and offers a gRPC service for flows and Prometheus metrics. Hubble Relay connects to the gRPC APIs of all servers and presents one API for the whole cluster, or for several clusters in a Cluster Mesh. The hubble command-line client talks either to Relay or to a local server. Hubble UI uses Relay to draw a service dependency map.

By default the Hubble API has node scope: an agent only sees the traffic that it handles itself, and the hubble binary inside each agent pod reads it through a local Unix domain socket. Relay is what turns these per-node views into a cluster-wide one, and it needs TCP port 4244 open on every node running Cilium so it can reach the servers through the hubble-peer service. If Relay logs connection refused errors while dialing hubble-peer, it cannot reach the agents' Hubble API.

The documentation frames Hubble around concrete questions. For service dependencies it answers which services talk to each other, how often and which HTTP calls are made. For monitoring it shows whether communication is failing and at which layer, for example DNS resolution problems, interrupted TCP connections or unanswered SYN requests. For application monitoring it can report rates of 4xx and 5xx responses and request latency percentiles, and for security observability it shows which connections were blocked by network policy and which services were reached from outside the cluster.

Flows can also become metrics. Hubble metrics describe the network behaviour of pods and are enabled by listing them in hubble.metrics.enabled, for example dns, drop, tcp, flow and httpV2, with labels controlled by context options such as sourceContext or labelsContext. They are exported under the hubble_ prefix on port 9965. They are separate from Cilium metrics, which describe the agent, Envoy and operator processes themselves and are switched on with prometheus.enabled.

Hubble is treated as a non-critical subsystem. When it fails to start, for example because the metric list names an unknown metric, the agent keeps running and forwarding traffic, and cilium status reports a warning rather than an outage.