Nybble™
← Back to modules

Fine-tuning & PEFT — Core

Full fine-tuning vs parameter-efficient methods: LoRA, QLoRA, adapters, prefix/prompt/P-tuning, freezing, rank and alpha, adapter merging, and when to fine-tune.

Medium50 questions

Sample questions

1

What is the core idea behind parameter-efficient fine-tuning (PEFT)?

  • Retrain the whole model but with a smaller learning rate
  • Train a few new or selected parameters, freezing the base
  • Distill the base model into a smaller student network
  • Quantize the base model to fit it on a smaller GPU

Why

PEFT freezes the bulk of the pretrained weights and trains only a small number of added or selected parameters, cutting memory and storage while keeping most of the base knowledge. It is not full retraining, distillation, or quantization — those are different techniques.

Hugging Face PEFT docsLoRA
2

How does LoRA modify a frozen weight matrix W during fine-tuning?

  • It replaces W entirely with a smaller randomly initialized matrix
  • It prunes the smallest-magnitude entries of W to save memory
  • It quantizes W to 4-bit and trains the quantization scales
  • It adds a low-rank update BA, only training A and B

Why

LoRA keeps W frozen and learns a low-rank additive update ΔW = BA (rank r ≪ dim), so only A and B are trained and stored. It doesn't replace, prune, or quantize W — those describe other methods.

LoRAHugging Face PEFT docs
3

What does QLoRA add on top of LoRA to fine-tune very large models on a single GPU?

  • It doubles the LoRA rank to compensate for lower precision
  • It trains LoRA adapters over a 4-bit quantized base
  • It distills the base model before applying LoRA
  • It fine-tunes only the embedding layer in full precision

Why

QLoRA freezes the base model in 4-bit (NF4) and trains LoRA adapters through it, so a huge model fits in far less memory while quality stays close to 16-bit fine-tuning. It doesn't rely on doubling rank, distillation, or embedding-only tuning.

QLoRALoRA
4

Why is full fine-tuning of a large model often impractical compared with PEFT?

  • It cannot change the model's behavior on the target task
  • It always overfits regardless of the dataset size
  • It requires storing optimizer state and gradients for every parameter
  • It is incompatible with GPUs and needs specialized hardware

Why

Full fine-tuning holds gradients and optimizer moments (e.g. Adam's two states) for all billions of parameters, which dominates memory; PEFT trains a tiny subset and sidesteps most of that. Full FT can change behavior, doesn't always overfit, and runs on standard GPUs.

LoRAHugging Face PEFT docs
5

A key practical advantage of LoRA is that a trained adapter is:

  • A full copy of the model weights fine-tuned for the task
  • A quantized version of the base that replaces the original
  • A prompt template stored alongside the tokenizer
  • A tiny file you swap per task without copying the base

Why

Because only A and B are trained, a LoRA adapter is megabytes, so one base model can serve many tasks by loading different adapters. It is not a full weight copy, a quantized base, or a prompt template.

LoRAHugging Face PEFT docs
Free account

Take the full module

These are the first few of 50 questions. A free account opens the rest as a scored drill.

  • Every question in this module
  • Instant feedback and supporting reading
  • Your score and progress, saved

Free · your email is used for progress only.

About Nybble™

The AI space moves fast.
Nybble™ is how you keep up — and stay sharp.

Snack

What happened. In two minutes.

The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.

Go to Snack
Stack

The concepts behind the headlines.

News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.

Go to Stack
Hack

Prove you actually get it.

Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.

Go to Hack