OpenTelemetry

• Key points are in the visual diagram above.

Agent stacks mix frameworks, models, and custom services. OTel lets you trace the entire request path in one system instead of siloed vendor dashboards.

OpenTelemetry is USB-C for observability — one plug on your code, many monitors on the other end.

Scenario

OTel trace spans: API gateway → agent orchestrator → OpenAI chat (1.2s) → Postgres tool (80ms) → response, all linked by trace_id.

Solution

In Eval Engineering & Observability, apply OpenTelemetry to this scenario: OTel trace spans: API gateway → agent orchestrator → OpenAI chat (1. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to OpenTelemetry (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where OpenTelemetry happens in the code.

1from opentelemetry import trace  # import dependencies2from opentelemetry.sdk.trace import TracerProvider  # import dependencies3from opentelemetry.sdk.trace.export import BatchSpanProcessor  # import dependencies4from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter  # import dependencies5
6provider = TracerProvider()7provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))8trace.set_tracer_provider(provider)

• Treating OpenTelemetry as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for opentelemetry

Quick recap — the most important points from this module.