Phoenix

• Key points are in the visual diagram above.

RAG agents fail silently when retrieval is bad. Phoenix links traces to retrieval quality, embedding drift, and hallucination patterns in one OSS tool.

Phoenix is an X-ray for RAG — it shows whether your agent's answers come from the right documents or imaginary sources.

Scenario

Phoenix trace shows retrieved chunks are about refunds when user asked about login — you fix the embedding model mismatch.

Solution

In Eval Engineering & Observability, apply Phoenix to this scenario: Phoenix trace shows retrieved chunks are about refunds when user asked about login — you fix the embedding model mismatch. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Phoenix (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Phoenix happens in the code.

1import phoenix as px  # import dependencies2from phoenix.otel import register  # import dependencies3from openinference.instrumentation.openai import OpenAIInstrumentor  # import dependencies4
5px.launch_app()6register()7OpenAIInstrumentor().instrument()

• Treating Phoenix as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for phoenix

Quick recap — the most important points from this module.