Generative AI
SearchStart here

Fine-tuning data, risks and shipping

  • Production
  • 20 min
  • Lesson 4 of 4 in Evaluation and model adaptation
  • Last verified
In this module
On this page

Before you start

You will be able to

  • Plan how to evaluate and ship a fine-tuned model: baseline to beat, regression and safety checks, held-out data, versioned adapters and rollback.
  • Name the main risks of fine-tuning (forgetting, overfitting, safety regression and leakage) and the check or mitigation for each.

Story

The change is done by four in the morning. Tobias reads the practice slips aloud, without a stumble, and Marta, thermos in hand, is very nearly smiling.

Merrow turns one slip over. It carries his FILED stamp from last week, from the day he boxed up the slips for the retraining pile.

“Flawless,” Merrow says, “on slips he has already met.” Marta puts the thermos down.

If the slips are not a fair test, which ones are?

Concept

Fine-tuning methods: SFT, LoRA, QLoRA and DPO covers how a model is tuned. This lesson covers what surrounds the training run: the data you feed it, what a tune can break, and the evidence you need before it goes live. A tune that trains without errors has proved nothing yet. It has only produced a candidate.

Why it matters

Choosing a method takes one decision. Many weak fine-tunes come from data that does not match production, a regression nobody tested for, or a score that was measured on the wrong set. The checks in this lesson are what separate “it trained” from “it is better than what we already have”.

Mental model

Treat a fine-tune as a release of your product, not as a research result. It needs the same things a code release needs: a baseline to compare against, tests, a canary and a rollback.

Where the analogy breaks: a code change usually affects the code you touched. A tune shifts behaviour across the whole model, so a regression can appear on tasks you never meant to change, which is why the regression suite exists.

Data matters most

  • Quality and diversity often matter more than volume, so a few thousand clean, deduplicated examples can beat a large noisy set. Start small and measure.
  • Match the production prompt format exactly, including the system prompt and chat templateChat templateThe exact message format a model was trained with. Training on a different format from production is a common cause of a model that looks good offline and worse live.Open the glossary entry. Training on one format and serving another is a common cause of a model that looks worse after tuning.
  • Hold out a test set and remove anything that overlaps your eval data (decontamination).
  • Label noise is learned as faithfully as signal.

Risks

  • ForgettingForgetting (catastrophic forgetting)The loss of general abilities a model was not trained to keep after fine-tuning. Mix in general data and re-run regression tests.Open the glossary entry. The model regresses on general abilities it was not trained to keep. Mitigate by mixing general data into the training set, lowering the learning rate or using fewer epochs, using a smaller adapter, and re-running the regression suite after every change.
  • OverfittingOverfittingA model memorising its training examples and generalising worse.Open the glossary entry. It memorises the training examples and generalises worse. Watch the held-out score during training, stop earlier when it stops improving, and add more varied examples.
  • Safety regression. Fine-tuning can weaken safety behaviour that the base model had; Qi et al. (2023) showed this for aligned models, even with benign data to a lesser degree. Run safety evals on the tuned model itself, because the base model’s results do not carry over.
  • Leakage. Training data that overlaps your evals makes scores meaningless. Decontaminate the training set against your evals and keep a held-out set the model never sees.

Evaluating and shipping a fine-tune

  1. Beat the best baseline. Compare against the base model with the best prompt, plus RAG if facts matter. If the tune does not clearly beat that, stay with the baseline.
  2. Task metrics on a held-out set, with the noise discipline from Evaluating LLM systems.
  3. Regression suite on general capabilities and safety checks.
  4. Human or judge comparison against the baseline on real inputs, with the judge validated.
  5. Deploy adapters versioned together with their data and hyperparameters. Canary first, and keep a rollback to the previous adapter or base.

No

Yes

Pretrained base model

SFT
prompt + ideal response

Preference tuning
RLHF or DPO

Beats best
prompt + RAG baseline?

Fix data and rerun,
or stay with prompt/RAG

Deploy adapter
canary, rollback

Diagram summary: A pretrained model goes through SFT and optional preference tuning, then is compared with the best prompt and RAG baseline. If it does not win, you fix the data or stay with the baseline; if it wins, you deploy the adapter with a canary and rollback.

Real-world application

A team distils a large model’s JSON-extraction behaviour into a small model with LoRA. Suppose the small model’s per-call price is one tenth of the large model’s (illustrative), so a pipeline of 5 million calls a month would cost about a tenth, plus the fallback calls. They validate the small model against the large model on a held-out set before cutover, keep the large model as a fallback for low-confidence cases, and watch the failure slices after release.

Interview perspective

Provenance: common industry pattern; no company-specific claim.

Expect “how many examples do you need” and “the fine-tuned model got worse at X, why”. Weak answers skip the prompt and RAG baseline, train on a different prompt format than production uses, or report a score from data that overlaps the training set.

Practice questions

Provenance: generated practice.

  • Basic: Name four risks of fine-tuning and one check for each.
  • Intermediate: Why must training examples use the same system prompt and chat template as production?
  • Advanced: After fine-tuning, accuracy rose on your task but fell on general tasks. Why, and what do you do?
  • Product: Decide whether to fine-tune a small model for a support-ticket classifier that a large prompted model already handles.

Activity

Do, notice, apply.

  1. Do. For the support-ticket classifier above, write the baseline you must beat, where the held-out set comes from, one regression check and one safety check.
  2. Notice. Which of your checks would fail first if training used bare questions while production adds a system prompt and a chat template?
  3. Apply. Write the rollback trigger: which slice regression makes you revert, and to what.

Homework

Write a fine-tuning spec: objective, data collection and cleaning, split and decontamination, method (full, LoRA or QLoRA, chosen as in the previous lesson), the baseline to beat, regression and safety evals, and the rollout and rollback plan.

Quick revision

  • Data quality beats volume; match the production prompt format; hold out a test set and decontaminate.
  • Four risks: forgetting, overfitting, safety regression, leakage.
  • Beat the prompt and RAG baseline before shipping.
  • Regression and safety checks, then a canary with versioned adapters and a rollback.

Check yourself

  • Checkpoint (answer aloud, no notes): List the evidence you would want before shipping a fine-tuned adapter, in order.
  • Re-test (nearby variant): Your fine-tune improved tone but hallucinated more facts. Which check should have caught it, and what do you do?
  • Misconception to avoid: “The task score went up, so it is ready to ship.”

Practice

Test what you just read. Quizzes and cards are grouped by module: Evaluation and model adaptation.

Check yourself6 questions on what this lesson set out to teach

Checking answers needs JavaScript. You can still read every question and option below.

0 of 6 answered

Keyboard: arrow keys move between options. Space or Enter selects the focused option.

ObjectivePlan how to evaluate and ship a fine-tuned model: baseline to beat, regression and safety checks, held-out data, versioned adapters and rollback.

  1. Question 1 of 6After fine-tuning, accuracy on your task rose, but a general-ability regression suite shows a drop. What is the most likely cause, and what is the next step?

    Choose one answer.

  2. Question 2 of 6A tuned model scores well in the notebook but worse than the base model in production. Training prompts were bare questions, while production adds a system prompt and a chat template. What is the fix?

    Choose one answer.

  3. Question 3 of 6A week after rollout, a canary shows a regression on one customer slice with the new adapter. Which earlier practice makes recovery fastest?

    Choose one answer.

ObjectiveName the main risks of fine-tuning (forgetting, overfitting, safety regression and leakage) and the check or mitigation for each.

  1. Question 4 of 6A team tunes an aligned model on a few thousand harmless support chats, and its task score is excellent. What should guide the release decision?

    Choose one answer.

  2. Question 5 of 6A retrained announcer scores almost perfectly on practice slips. Later you learn about half of those slips were in the training pile. What does the score tell you?

    Choose one answer.

  3. Question 6 of 6During training the loss on the training examples keeps falling, but accuracy on the held-out set stops improving and then drops. What is happening, and what is a sensible response?

    Choose one answer.

Build with this

On this page