Collector Pipelines and Connectors: OTCA practice questions

A pipeline is the path telemetry takes through the Collector, from reception through optional processing to export. Each pipeline has a data type, which is traces, metrics or logs, and the type is part of the pipeline identifier, as in metrics or metrics/internal. Every receiver, processor and exporter used in a pipeline has to support that type. If one does not, the Collector reports a signal-not-supported error when it loads the configuration, so the mistake surfaces at startup rather than at runtime.

Inside a pipeline, data from all listed receivers flows into the first processor, then through each processor in the order written, and the last processor hands it to the exporters. That hand-off is a fan-out: every exporter receives its own copy of every data element. A processor may also drop data, which is how sampling and filtering work. The batch processor guidance adds an ordering rule: put memory_limiter first and batch after any sampling, so batching happens after data has been dropped.

Sharing components across pipelines follows different rules for each kind. A receiver listed in several pipelines runs as a single instance that feeds a fan-out consumer, and the hand-off is a synchronous call. If a processor in one of those pipelines blocks, the other pipelines stop receiving that receiver's data and the receiver stops accepting new data. Exporters can likewise be shared, and several pipelines can send to the same exporter. Processors are never shared: each pipeline that references a processor gets its own instance with its own state, built from the same configuration. A single pipeline must not list the same processor twice.

Connectors link pipelines. A connector is listed as an exporter at the end of one pipeline and as a receiver at the start of another, and the two sides may carry the same or different signal types. Each connector supports only specific pairs. The count connector turns spans, metrics or logs into count metrics. If the traces pipeline exports only to count, the original spans go nowhere unless you also add a trace exporter. The span metrics connector, now typed span_metrics with spanmetrics kept as a deprecated name, consumes traces and emits request, error and duration metrics. The forward connector has no settings and simply merges or forks pipelines of one type. The routing connector sends data to different pipelines based on OTTL conditions. By default a match moves data to the target and removes it from later routes, while the copy action leaves it available to later routes, and default_pipelines catch anything that matches nothing.

Seen together, these rules explain most pipeline designs: receivers fan data into pipelines, processors shape it in order, exporters each get a copy, and connectors let one pipeline's output become another pipeline's input.