LangSmith

LangGraph/LangChain apps need first-class debugging. LangSmith shows every node input/output, lets you annotate failures, and runs regression evals in CI.

LangSmith is a flight recorder plus test lab for your LangChain apps — replay crashes and run simulations before passengers board.

Scenario

Upload 50 support tickets as a dataset; run agent v2; LLM-as-judge scores resolution quality; block deploy if score drops >5%.

Solution

In Agent Evaluation & Observability, apply LangSmith to this scenario: Upload 50 support tickets as a dataset; run agent v2; LLM-as-judge scores resolution quality; block deploy if score drops >5%. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to LangSmith (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where LangSmith happens in the code.

1import langsmith as ls  # import dependencies2from langsmith.evaluation import evaluate  # import dependencies3
4@ls.testing.traceable5def my_agent(inputs):  # define a reusable function6    return app.invoke(inputs)  # return the result7
8results = evaluate(my_agent, data="support-golden-set", evaluators=[correctness])

• Treating LangSmith as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for langsmith

Quick recap — the most important points from this module.