Fine Tuning

Prompt engineering has limits — very long system prompts, inconsistent formatting, domain-specific jargon. Fine-tuning bakes behavior into model weights: consistent tone, specialized vocabulary, specific output formats, and improved performance on your exact task.

Prompt engineering is giving instructions to a generalist consultant each meeting. Fine-tuning is hiring that consultant full-time and training them on your company's processes — they internalize your way of working.

Scenario

Your support bot needs to respond in your company's specific tone and always include a ticket number. After 1000 examples of ideal responses, fine-tuning produces consistent formatting that prompt engineering couldn't reliably achieve.

Solution

In Advanced AI, apply Fine Tuning to this scenario: Your support bot needs to respond in your company's specific tone and always include a ticket number. Identify the inputs, run the technique, validate the output, and note one thing you would monitor in production.

Do this before moving to the next module — reading alone is not enough.

Open the Code Walkthrough below and run it locally. Change one parameter related to Fine Tuning (e.g. model, temperature, top_k, or tool name), observe the difference in output, and write 2–3 sentences explaining what changed.

Highlighted lines show where Fine Tuning happens in the code.

1# OpenAI fine-tuning data format (JSONL)2# {"messages": [3#   {"role": "system", "content": "You are a support agent."},4#   {"role": "user", "content": "My order is late"},5#   {"role": "assistant", "content": "I apologize for the delay. Ticket #12345. Let me check..."}6# ]}7
8from openai import OpenAI  # import dependencies9client = OpenAI()  # create API client10
11# Upload training file12# file = client.files.create(file=open("training.jsonl", "rb"), purpose="fine-tune")13# job = client.fine_tuning.jobs.create(training_file=file.id, model="gpt-4o-mini-2024-07-18")14
15# Use fine-tuned model16response = client.chat.completions.create(  # call the API17    model="ft:gpt-4o-mini:my-org:abc123",18    messages=[{"role": "user", "content": "My order is late"}],19)

• Fine-tuning when prompt engineering or RAG would suffice
• Low-quality training data producing worse results
• Fine-tuning for factual knowledge (use RAG instead)
• Not evaluating fine-tuned model against baseline before deploying

Quick recap — the most important points from this module.