Prompt Versioning

Prompt tweaks silently break production. Versioning lets you compare v3 vs v4 on eval sets before full rollout.

Recipe versioning in a restaurant — when a dish changes, you keep the old recipe card in case customers complain.

Scenario

Support bot quality drops after deploy. Roll back prompt from v7 to v6 in one config change while investigating.

Solution

In Production Agent Engineering, apply Prompt Versioning to this scenario: Support bot quality drops after deploy. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Prompt Versioning (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Prompt Versioning happens in the code.

1# Prompt Versioning — minimal example2from openai import OpenAI3
4client = OpenAI()  # create API client5
6# Ask the model to explain this topic7response = client.chat.completions.create(  # core API call for Prompt Versioning8    model="gpt-4o-mini",9    messages=[10        {"role": "system", "content": "You explain prompt versioning clearly."},11        {"role": "user", "content": f"What is prompt versioning?"},12    ],13    temperature=0,14)15print(response.choices[0].message.content)  # show output for debugging

• Treating Prompt Versioning as a black box without evaluation
• Ignoring cost and latency in production
• Skipping error handling for prompt versioning

Quick recap — the most important points from this module.