Agent System Design

Companies hire agent engineers who can whiteboard production architectures, not just call APIs. This module prepares you for 45-minute design rounds focused on reliability and tradeoffs.

Designing a restaurant kitchen: menu (capabilities), stations (agents/tools), order tickets (state), health inspector rules (guardrails), and rush-hour staffing (scaling).

Scenario

Prompt: 'Design a customer support agent for 1M users.' You scope: tier-1 automation with KB RAG, CRM tools, escalation to human, p95 < 5s, $0.02/ticket target — diagram ingestion, router, agent runtime, observability, and fallback when LLM is down.

Solution

In Interview & System Design, apply Agent System Design to this scenario: Prompt: 'Design a customer support agent for 1M users. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Agent System Design (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Agent System Design happens in the code.

1# Agent System Design — minimal example2from openai import OpenAI3
4client = OpenAI()  # create API client5
6# Ask the model to explain this topic7response = client.chat.completions.create(  # core API call for Agent System Design8    model="gpt-4o-mini",9    messages=[10        {"role": "system", "content": "You explain agent system design clearly."},11        {"role": "user", "content": f"What is agent system design?"},12    ],13    temperature=0,14)15print(response.choices[0].message.content)  # show output for debugging

• Treating Agent System Design as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for agent system design

Quick recap — the most important points from this module.