Monitoring

Agents fail silently: hallucinated answers look like success, tool timeouts return empty strings, costs spike from runaway loops. Without observability, you're flying blind in production.

Monitoring is the instrument panel in a cockpit — altitude, fuel, engine warnings. You don't wait for passengers to scream before checking the gauges.

Scenario

Langfuse trace shows a 45s request: 30s spent in a retried web-scrape tool. Alert fires on p95 > 15s. Engineer adds timeout + fallback, latency drops to 6s.

Solution

In Production Agent Engineering, apply Monitoring to this scenario: Langfuse trace shows a 45s request: 30s spent in a retried web-scrape tool. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Monitoring (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Monitoring happens in the code.

1from opentelemetry import trace  # import dependencies2from opentelemetry.trace import Status, StatusCode  # import dependencies3
4tracer = trace.get_tracer("agent")5
6async def call_llm(prompt: str) -> str:7    with tracer.start_as_current_span("llm.call") as span:8        span.set_attribute("model", "gpt-4o-mini")9        span.set_attribute("prompt.length", len(prompt))10        try:11            result = await llm.invoke(prompt)12            span.set_attribute("tokens.output", result.usage.completion_tokens)13            return result.content  # return the result14        except Exception as e:15            span.set_status(Status(StatusCode.ERROR, str(e)))16            raise

• Treating Monitoring as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for monitoring

Quick recap — the most important points from this module.