Workers

Separating API from compute lets you scale each independently, retry failed agent runs without the user waiting, and run resource-heavy tasks (browser automation, large PDF parsing) on machines tuned for that workload.

Workers are kitchen staff behind the counter — the front desk (API) takes orders fast; the kitchen (workers) does the slow cooking without making customers stand at the register.

Scenario

Three Celery workers consume from `agent.tasks` queue. Each runs a LangGraph agent for document summarization. On failure, Celery retries 3× with exponential backoff, then moves to DLQ for manual review.

Solution

In Production Agent Engineering, apply Workers to this scenario: Three Celery workers consume from `agent. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Workers (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Workers happens in the code.

1from celery import Celery  # import dependencies2
3app = Celery("agent", broker="redis://localhost:6379/0")4
5@app.task(bind=True, max_retries=3, default_retry_delay=60)6def summarize_document(self, doc_id: str) -> dict:  # define a reusable function7    try:8        text = fetch_document(doc_id)9        summary = run_summarization_agent(text)10        save_summary(doc_id, summary)11        return {"doc_id": doc_id, "status": "done"}  # return the result12    except TransientError as exc:13        raise self.retry(exc=exc)

• Treating Workers as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for workers

Quick recap — the most important points from this module.