Queues

Long-running agent tasks (research reports, batch evals, multi-step workflows) shouldn't block HTTP responses. Queues absorb traffic spikes, survive consumer crashes via message persistence, and let you scale workers independently of the API tier.

A queue is a ticket counter at a deli — customers take a number and leave; staff call numbers when ready. No one crowds the counter, and orders aren't lost if a cook steps out.

Scenario

User submits 'generate quarterly report' via API → API enqueues job `{job_id, user_id, query}` → returns 202 Accepted → worker picks up job, runs 5-minute agent pipeline, stores result in S3, notifies user via webhook.

Solution

In Production Agent Engineering, apply Queues to this scenario: User submits 'generate quarterly report' via API → API enqueues job `{job_id, user_id, query}` → returns 202 Accepted → worker picks up job, runs 5-minute agent pipeline, stores result in S3, notifies user via webhook. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Queues (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Queues happens in the code.

1import json  # import dependencies2import redis  # import dependencies3
4r = redis.Redis()5
6def enqueue_agent_job(job_id: str, payload: dict) -> None:  # define a reusable function7    r.lpush("agent:jobs", json.dumps({"job_id": job_id, **payload}))8
9def worker_loop():  # define a reusable function10    while True:11        _, raw = r.brpop("agent:jobs", timeout=5)12        job = json.loads(raw)13        try:14            result = run_agent_pipeline(job)15            store_result(job["job_id"], result)16        except Exception:17            r.lpush("agent:dlq", raw)  # dead-letter queue

• Treating Queues as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for queues

Quick recap — the most important points from this module.