Key takeaways

The through-line

Fine-tuning teaches behaviour, format and style — not facts. Almost every disappointing fine-tune started as a knowledge problem that should have been solved with retrieval, and the first lesson of the course is the decision framework that prevents it.

The ideas that carry

• Decide before you train. Prompting, RAG and fine-tuning solve different problems. Fine-tuning wins on consistent output format, domain tone, latency (a smaller tuned model) and behaviour that resists prompting.
• LoRA works because weight updates are low-rank. Freeze the base, train small adapter matrices, and get most of full fine-tuning's benefit at a fraction of the memory. r and alpha are the dials; which modules you target matters more than people expect.
• QLoRA made a single GPU enough — 4-bit NF4 base weights, LoRA in higher precision on top. The technique that moved fine-tuning from a cluster to a workstation.
• Data quality dominates data quantity. A thousand carefully-built examples routinely beat fifty thousand scraped ones. Template and chat-format correctness is a silent killer here.
• Synthetic data is legitimate and needs the same scrutiny — diversity, filtering, and honesty about the teacher model's licence and its biases.
• Measure before and after on the same held-out set, and check for regressions on general capability. A model that got better at your task and worse at everything else is a real outcome.
• DPO simplified preference tuning by removing the separate reward model, which is why it displaced classic RLHF for most teams.
• Multi-adapter serving is the deployment payoff: one base model in memory, many adapters swapped per request.

What you can now do

Justify fine-tuning against the alternatives. Build a dataset and get the chat template right. Run LoRA/QLoRA on accessible hardware. Evaluate honestly and ship with a model card and a serving path.

If one thing sticks

If the answer is "the model doesn't know our data", you want retrieval. Fine-tune when the answer is "the model doesn't behave the way we need".