Memory Design

Agents without memory feel forgetful; agents with bad memory leak stale facts and blow token budgets. Interviews test your tiered memory architecture.

Human memory: sticky note on desk (short-term), journal summary (compressed), filing cabinet (long-term search) — not one infinite notebook.

Scenario

Support agent remembers user prefers email contact (profile DB), summarizes last three tickets (session), retrieves similar resolved cases (vector) — without sending entire history every turn.

Solution

In Interview & System Design, apply Memory Design to this scenario: Support agent remembers user prefers email contact (profile DB), summarizes last three tickets (session), retrieves similar resolved cases (vector) — without sending entire history every turn. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Memory Design (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Memory Design happens in the code.

1# Memory Design — minimal example2from openai import OpenAI3
4client = OpenAI()  # create API client5
6# Ask the model to explain this topic7response = client.chat.completions.create(  # core API call for Memory Design8    model="gpt-4o-mini",9    messages=[10        {"role": "system", "content": "You explain memory design clearly."},11        {"role": "user", "content": f"What is memory design?"},12    ],13    temperature=0,14)15print(response.choices[0].message.content)  # show output for debugging

• Treating Memory Design as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for memory design

Quick recap — the most important points from this module.