Fine-tuning and LoRA

The video is in Italian; this page is the written version in English.

Fine-tuning takes an already trained model and trains it a bit more, on a small curated dataset, to specialise it. The general practitioner going through a specialist course: no general medicine is forgotten, expertise is added.

The question to ask first

Fine-tuning is fascinating and almost always the wrong first move. This table is worth being blunt about:

What you want	Fine-tuning?	Try first
A consistent tone or style	yes	—
A rigid output format	maybe	examples in the prompt
Adding specific knowledge	rarely	RAG
Data that changes over time	no	RAG
Doing better at a precise task	yes	prompt engineering

Rule of thumb: prompt first, then RAG, and only then fine-tuning. The reason is practical: knowledge baked into the weights is frozen there, updating it costs another training run, and you risk degrading abilities the model already had. An external archive is updated by changing a file.

LoRA: renovating without demolishing

Full fine-tuning retrains every weight: for an 8-billion-parameter model that means laboratory hardware. LoRA — Low-Rank Adaptation — changes the game.

The analogy is renovating a house. Instead of demolishing and rebuilding, you freeze the load-bearing structure and add targeted changes: a wall here, wiring there.

Technically: the original weights stay intact and frozen, and alongside them two small matrices are trained to modify their behaviour. Trainable parameters collapse — often under 1% of the total — and so does the memory needed.

The result is a file of a few dozen megabytes, the adapter, applied to the original model at use time. You can keep several and swap them like lenses.

QLoRA adds one more step: the base model is quantised to 4 bits and the adapter trained on top. That combination is what brought fine-tuning to consumer hardware.

What you actually need

The bottleneck is not how much data you have, it is how good it is. Five hundred well-written, internally consistent examples beat five thousand scraped ones. The examples must look like what you want: if you need a certain tone, every example must have that tone, no exceptions.

The classic failure is overfitting: the model memorises the examples instead of generalising. You spot it because it does beautifully on training data and poorly on everything else. Antidotes: hold out a slice of data for checking, stop early, do not overdo the training epochs.

Keep an eye on catastrophic forgetting too: over-specialise and the model can lose general abilities it had. LoRA suffers less than full fine-tuning, precisely because the original weights stay put.

How to tell whether it worked

The part missing from nearly every tutorial: once training ends, how do you establish the model improved rather than merely changed? Three measures, and all three are needed.

Measure	How	What it answers
Task performance	On examples never seen in training	Did it learn or memorise?
Against the base	Same examples, starting model	Did fine-tuning add anything?
Regressions	Generic prompts outside the domain	Did it lose abilities it had?

The third is the one people skip and the one that hurts: a beautifully specialised model may have become useless at everything else. Twenty general prompts, before and after, are enough to notice.

The operating rule: hold out 10-20% of examples from training and use them only to measure. If results are excellent on training material and mediocre on the held-out set, you have a textbook case of overfitting.

One model, many lenses

A practical consequence of the mechanism: since the original weights stay intact and the adapter is a separate file, you can keep several adapters and swap them live on top of a single model loaded in memory.

                      ┌── "corporate tone" adapter
base 8B model    ─────┼── "data extraction" adapter
(loaded once)         └── "ticket triage" adapter

On limited hardware the advantage is obvious: instead of three specialised models of several gigabytes each, one model plus three files of a few dozen megabytes. It is also the right way to think about specialisation — not “one model per task” but “one model, many lenses”.

What it actually costs

Tutorials show the notebook running in twenty minutes and let you believe that is the work. The real time is distributed very differently.

preparing and cleaning data   ████████████████████  60-70%
training                      ████                  10%
evaluating and comparing      ██████                20%
exporting and integrating     ██                    5%

And it is not one round. The first attempt almost always serves to discover the data had a problem — inconsistent examples, a wrong format, cases too similar to each other. Budget three rounds before you have something usable.

One way to shorten it: start small. A hundred examples and few epochs produce a mediocre model, but in half an hour they tell you whether the path is right, whether the format holds, and whether the evaluation you prepared measures what you care about. Better to find a design error on a hundred examples than on five thousand.

The checklist before starting

Five questions that save days.

Question	If the answer is no
Have I tried better prompts and they fall short?	Go back to the prompt: costs nothing, often enough
Is the problem how it answers, not what it knows?	If it is knowledge, you need RAG
Do I have at least 300-500 consistent examples?	Collect them first: no data, no start
Have I held out examples for verification?	Do it now, or you will not know if it improved
Can I say in one sentence what must change?	If you cannot say it, the model will not guess

The last is the most underrated. “I want better answers” is not a trainable goal. “I want answers of at most three sentences, always citing the reference clause, with no pleasantries” is — and it is also the criterion you will measure against.

In short

Concept	In one line
Fine-tuning	Training an already trained model further, on targeted data
LoRA	Freezes the original weights and trains two small matrices alongside
QLoRA	LoRA on a 4-bit quantised base model: runs on ordinary hardware
Adapter	The lightweight output file, applied and swapped at will
Right order	Prompt → RAG → fine-tuning, never the reverse
Evaluation	Three measures: fresh examples, base comparison, regressions
Multiple adapters	One loaded model, many interchangeable specialisations
The real time	Two thirds on data, one tenth on training
Before starting	Five questions: one “no” and fine-tuning is not the move