Tree of Thoughts

ReAct follows a single reasoning path — if the first approach is wrong, it may waste many steps backtracking. ToT explores alternatives in parallel, evaluates each branch, and commits to the most promising — dramatically improving success on complex reasoning tasks.

ToT is like a chess player considering multiple moves ahead — not just playing the first reasonable move, but evaluating several lines of play before choosing the best one.

Scenario

Task: 'Debug why API latency spiked.' Branch A: check server logs. Branch B: check database queries. Branch C: check network. Evaluator scores A highest (recent deploy correlates). Expands A: check deploy diff → finds memory leak.

Solution

In Agent Design Patterns, apply Tree of Thoughts to this scenario: Task: 'Debug why API latency spiked. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Tree of Thoughts (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Tree of Thoughts happens in the code.

1def tree_of_thoughts(problem: str, k: int = 3, depth: int = 3) -> str:  # define a reusable function2    def generate_thoughts(state: str, k: int) -> list[str]:  # define a reusable function3        response = client.chat.completions.create(  # call the API4            model="gpt-4o",5            messages=[{"role": "user", "content": f"Generate {k} different approaches for: {state}"}],6            temperature=0.7,7        )8        return response.choices[0].message.content.split("\n")  # return the result9
10    def evaluate(state: str, thought: str) -> float:  # define a reusable function11        response = client.chat.completions.create(  # call the API12            model="gpt-4o",13            messages=[{"role": "user", "content": f"Score 0-1 how promising: {thought} for {state}"}],  # key line for Tree of Thoughts14            temperature=0,15        )16        return float(response.choices[0].message.content.strip())  # return the result17
18    best_path = search(problem, k, depth, generate_thoughts, evaluate)  # key line for Tree of Thoughts19    return execute_path(best_path)  # return the result

• Using ToT for simple tasks — massive cost overhead
• Too many candidates (k>5) — exponential cost growth
• No depth limit — infinite branching
• Unreliable thought evaluation — pruning good branches
• Not comparing cost/benefit vs simpler ReAct

Quick recap — the most important points from this module.