Key takeaways
The through-line
RAG is a retrieval system with a language model attached, and almost every RAG failure in production is a retrieval failure wearing a generation costume. The course's arc is exactly that realisation: the interesting engineering is upstream of the model.
The ideas that carry
- RAG is one of four options, not the default. Fine-tuning, long context and tool use solve overlapping problems. RAG earns its place when knowledge is large, changing, or must be cited.
- Chunking is the highest-leverage decision you will make and the one most often made by accident. Chunk boundaries decide what can ever be retrieved together.
- Dense and sparse retrieval fail differently, which is why hybrid wins. Embeddings capture meaning and miss exact identifiers; BM25 does the reverse. Fusion beats either alone on real corpora.
- Reranking is where cheap wins live. Retrieve broadly with a fast index, then spend a cross-encoder on the top candidates. Precision at the top is what the generator actually sees.
- Query transformation fixes the mismatch between how people ask and how documents are written. Rewriting, decomposition and HyDE all attack the same gap.
- If you can't measure retrieval separately from generation, you can't debug either. Recall@k and nDCG on retrieval; faithfulness and answer relevance on generation. The triad exists so a bad answer can be attributed.
- Grounded generation is a context-assembly problem. Ordering, deduplication, citation scaffolding and an explicit "say you don't know" path do more for trustworthiness than any prompt phrasing.
- Access control has to live in the retrieval layer. A permission check after generation is already too late — the leak happened when the chunk entered the context.
What you can now do
Build RAG from scratch and know what each framework abstraction is hiding. Ship pgvector-backed retrieval. Evaluate a pipeline component by component. Decide, with evidence, whether a problem wants RAG, fine-tuning, long context or a tool.
If one thing sticks
When the answer is wrong, look at what was retrieved before you look at the prompt. It is almost always the retrieval.