LoRA math and rank/alpha, QLoRA internals (NF4, double quant, paged optimizers), DoRA, AdaLoRA, LoRA+, multi-adapter serving, quantization interplay, and forgetting trade-offs.
Why
Because the effective scale is α/r, raising r while keeping α/r roughly constant keeps the update magnitude stable, so you can sweep rank without re-tuning learning rate. It doesn't change freezing, initialization, or quantization.
Why
Double quantization compresses the per-block quantization constants (scales) with a second quantization step, shaving extra bytes per parameter. It is not 2-bit weights, activation quantization, or adapter quantization.
Why
DoRA splits each weight into a magnitude scalar and a direction vector, applying LoRA to the direction and learning the magnitude separately, which narrows the gap to full fine-tuning. It isn't a sparse/dense, q/v, or encoder/decoder split.
Why
AdaLoRA writes ΔW ≈ PΛQ and prunes the least-important singular values, moving rank budget to modules that need it. It doesn't merge each step, change quantization on the fly, or replace attention.
Why
LoRA+ shows that using a higher learning rate for B than for A improves convergence and final quality, since the two matrices play asymmetric roles. It's not about differing ranks, quantization, or dropout.
These are the first few of 50 questions. A free account opens the rest as a scored drill.
Free · your email is used for progress only.
About Nybble™
What happened. In two minutes.
The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.
Go to SnackThe concepts behind the headlines.
News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.
Go to StackProve you actually get it.
Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.
Go to Hack