Agentic AI Notebook
Back to Roadmap
Phase 19

Eval Engineering & Observability

How to prove an agent works: datasets, LLM-as-judge, online/offline eval, benchmarks, agent-specific OpenTelemetry, and regression suites.

18 modules

Estimated read time is shown per module based on lesson length.

Modules

1.Observability~2 min read
2.LangSmith~2 min read
3.Phoenix~2 min read
4.OpenTelemetry~2 min read
5.GenAI Semantic Conventions~2 min read
6.Agent Metrics~2 min read
7.W&B~2 min read
8.LLM Evaluation~2 min read
9.Agent Evaluation~2 min read
10.Tool Evaluation~3 min read
11.Trajectory Evaluation~2 min read
12.Eval Datasets~2 min read
13.LLM-as-Judge~2 min read
14.Online vs Offline Eval~3 min read
15.Eval-Driven Development~2 min read
16.Agent Benchmarks~2 min read
17.Hallucination Detection~2 min read
18.Regression Testing~2 min read

Phase 19 Glossary

50 terms · search to look up

Navigation

← Multi-Agent SystemsAgent Security & Governance →