Collector Deployment Patterns and Kubernetes: OTCA practice questions

The Collector ships as a single binary, and the documentation describes several ways to place it. In the simplest arrangement there is no Collector at all: each SDK exports straight to a backend. That is quick to set up and has no extra moving parts, but any change to collection or processing means changing application code, the application becomes coupled to the backend, and each language offers only a limited set of exporters.

In the agent pattern, the application sends OTLP to a Collector running next to it or on the same host, typically as a sidecar or a DaemonSet in Kubernetes. The docs list its strengths as being easy to start and giving a clear one-to-one mapping between application and Collector, and its weaknesses as limited scalability and inflexibility for complex or evolving setups.

In the gateway pattern, applications or agents send to a single OTLP endpoint served by one or more standalone Collector instances, often one endpoint per cluster, data center or region. Its benefits are separation of concerns, such as credentials managed in one place, and central policy for filtering or sampling. The costs are another component that can fail, added latency when Collectors are chained, and higher resource use. When data must reach one particular Collector, as tail sampling needs, the docs describe a two-tier gateway: the first tier runs the load-balancing exporter so that every span of a trace lands on the same second-tier instance.

On Kubernetes there are two official installation paths. The OpenTelemetry Operator, which needs cert-manager installed first, manages Collectors through the OpenTelemetryCollector custom resource and also handles auto-instrumentation. Its spec.mode can be deployment, daemonset, statefulset or sidecar, and deployment is the default. Sidecar mode injects the Collector into pods that carry the sidecar.opentelemetry.io/inject annotation, which belongs on the pod template rather than on the Deployment object. The Helm chart is the other path. It requires mode to be set to daemonset, deployment or statefulset and offers presets that wire up RBAC, volumes and components. For example, the cluster metrics preset is recommended with a single-replica deployment, because several copies of the cluster receiver would emit duplicate data.

Whatever the topology, the Kubernetes attributes processor is the key enrichment step for application telemetry. It watches pods through the Kubernetes API and, by default, matches incoming data to a pod by the connection IP, then adds resource attributes such as k8s.pod.name and k8s.namespace.name. Agents should restrict discovery to their own node with filter.node_from_env_var and a node name injected through the downward API. When a gateway sits behind agents, the agents can run the processor in passthrough mode so the original pod IP travels with the data.