What is fine-tuning?

The short answer

Fine-tuning is taking a model that already works and continuing to train it on your examples, so it adjusts its behaviour toward the pattern you showed it. You are not adding facts. You are changing how it responds.

That distinction decides almost every fine-tuning question. If your complaint is that the model does not know something, fine-tuning is the wrong tool. If your complaint is that it knows but answers in the wrong shape, you are in the right place.

What it actually changes

A model has weights, which encode both what it absorbed during training and how it tends to respond. Fine-tuning nudges those weights using examples of input and desired output, usually a few hundred to a few thousand pairs.

What moves reliably: format, tone, structure, adherence to a house style, consistency on a narrow repetitive task, and the ability to stop producing the preamble nobody asked for. What does not move reliably: knowledge of facts you showed it a handful of times. The model will absorb the shape of your examples long before it absorbs their content.

Fine-tuning, RAG, or just a better prompt

These solve three different problems and teams routinely reach for the most expensive one first.

Prompting changes behaviour with no training, instantly, and is reversible. RAG gives the model information it did not have, at question time, and stays current as your data changes. Fine-tuning changes default behaviour permanently and goes stale the moment your requirements move.

The practical rule: exhaust prompting, then reach for retrieval, then consider fine-tuning. We wrote about the retrieval side in what RAG is, and the split matters because a knowledge problem dressed as a behaviour problem can burn a quarter.

When it is the right call

Four situations make it genuinely worth it. A narrow, high volume task where the same shape repeats thousands of times a day and every token of prompt overhead is a real cost. A house format that prompting keeps drifting away from. A specialised vocabulary or classification scheme the base model keeps approximating.

And latency or cost pressure, where a smaller fine-tuned model can match a larger general one on your specific task for a fraction of the price per request. That last one is the most defensible business case, and the least discussed.

When it is the expensive wrong call

When you want the model to know your products, your prices, your policies, or anything that changes. Bake those into weights and you have built a system that is confidently wrong the day after a price update.

When you have fifty examples. That is a prompt, not a dataset. When nobody can articulate what the desired output looks like, in which case you have a specification problem and fine-tuning will faithfully learn your confusion. And when the base model has not been given a fair chance with a serious prompt, which is most of the time.

What the data has to look like

Input and output pairs that represent the job as it actually arrives, not as you wish it arrived. Consistency matters more than volume: two thousand examples where the format wobbles teach the model to wobble.

Include the hard cases and the ones where the correct output is a refusal or an escalation. A dataset made only of clean successes produces a model that never says it does not know, which is the failure mode that costs the most in production.

Assume the data preparation is the project. Teams budget for the training run, which is cheap and fast, and are surprised by the weeks of labelling that precede it.

What it costs, and what it costs to keep

The training run itself is usually the smallest line. The real costs are preparing the dataset, evaluating whether the result is actually better, and the fact that you now own a model artifact that has to be versioned, redeployed, and re-evaluated every time the base model updates or your requirements shift.

Serving may be cheaper per request, which is often the whole point. But you have traded a variable cost for a fixed maintenance obligation, and that trade only pays at volume.

How to know it worked

Before you tune anything, build the evaluation set. Otherwise you will compare the new model to the old one by reading a dozen outputs and preferring the one you spent money on. We covered how to build that in LLM evals.

Hold out cases the model never saw. Compare against the base model with your best prompt, not against the base model with a lazy prompt, because that comparison is how fine-tuning gets credit it did not earn.

And check for regression outside the target task. A model tuned hard on one format often gets worse at everything adjacent, which shows up later as a mysterious drop in a flow nobody was testing.

The order to try things in

Write a serious prompt with examples in it. Measure. Add retrieval if the gap is knowledge. Measure again. Only then consider fine-tuning, and only if you can name the specific behaviour that is still wrong and show it on a reference set.

Most teams that ask us about fine-tuning have a retrieval problem or an unmeasured system. Both are cheaper to fix and both have to be fixed anyway, which is why we work in that order when we build production AI, end to end.