Skip to content
CNPEObservability and OperationsSingle nodeMedium

Monitoring and alerting: ServiceMonitor, alert and recording rule

Scenario cnpe-obs-001 · about 18 minutes · 10 points · 5 grader checks

Get the shop API into the platform Prometheus: scrape it with a ServiceMonitor, alert on its 5xx ratio and precompute its request rate with a recording rule.

  • prometheus
  • servicemonitor
  • prometheusrule
  • alerting
  • recording-rules
  • observability

What you will practise

  • Write a ServiceMonitor that matches a Prometheus selector
  • Write alerting and recording rules in a PrometheusRule
  • Confirm targets and rules through the Prometheus HTTP API

Start this lab

With the CLI installed and signed in, this command provisions a single node cluster on your Mac, applies the scenario setup and prints the task below.

goldenkube start cnpe-obs-001

Task

The task text exactly as the CLI shows it. Solve it on the cluster, then run goldenkube submit.

The Prometheus "platform" in namespace "monitoring" (Prometheus Operator) scrapes ServiceMonitors and loads PrometheusRules from every namespace, but only objects with the label release=platform. In namespace "shop" the Deployment "api" (2 replicas) exposes metrics on /metrics through the Service "api", port name "http". 1. Create a ServiceMonitor "api" in "shop" that scrapes that Service port every 15s on /metrics. 2. Create a PrometheusRule "api-alerts" in "shop" with a rule group "api" containing: - the alert "APIHighErrorRate" firing when more than 5% of http_requests_total over 5 minutes have a 5xx status, for 5m, with the label severity=page; - the recording rule "job:http_requests:rate5m" for sum by (job) (rate(http_requests_total[5m])). 3. Prometheus must scrape both pods successfully and have both rules loaded.

Requirements and details

Certification
CNPE · Certified Cloud Native Platform Engineer
Domain
Observability and Operations
Difficulty
Medium
Cluster topology
Single node
Cluster features
None beyond the base cluster
Recommended time
18 minutes
Points
10
Grader checks
5
Kubernetes version
1.35
Minimum CLI version
0.1.0
Scenario version
1
Published

Grading runs locally: 5 state checks read your cluster when you submit; the scenario passes when every check passes. Hints, the reference solution and the check definitions are available only inside the CLI, not on this page.

Install the CLICreate a free account

More Observability and Operations labs

CNPEObservability and OperationsMedium

Tracing: add an OTLP traces pipeline to the OpenTelemetry Collector

The platform OpenTelemetry Collector only handles metrics. Give it an OTLP traces pipeline with the right processors, expose it, and prove it with generated spans.

Single node · ~18 min · 9 points

  • opentelemetry
  • otel-collector
  • tracing
  • otlp
  • telemetrygen
  • observability

goldenkube start cnpe-obs-002

CNPEObservability and OperationsHard

Logging: collect one namespace's pod logs with an OpenTelemetry DaemonSet

Ship only the shop namespace's container logs: run the OpenTelemetry Collector as a DaemonSet with the filelog receiver on the node's pod log files and print the records with the debug exporter.

Single node · ~18 min · 9 points

  • opentelemetry
  • logging
  • filelog
  • daemonset
  • hostpath
  • observability

goldenkube start cnpe-obs-003

CNPEObservability and OperationsMedium

Incident response: roll back a bad release and fix an OOM-killed workload

Two workloads in namespace checkout are degraded: diagnose a failed rollout and an OOM-killed container from events and status, remediate both in place and record the findings in an incident ConfigMap.

Single node · ~14 min · 10 points

  • incident-response
  • rollout
  • rollback
  • oomkilled
  • troubleshooting
  • observability

goldenkube start cnpe-obs-004

Independent community project, not affiliated with or endorsed by CNCF or The Linux Foundation. Lab results are practice feedback, not an exam score.