Building RAG Systems

Course wrap-up

Key takeaways

The through-line

RAG is a retrieval system with a language model attached, and almost every RAG failure in production is a retrieval failure wearing a generation costume. The course's arc is exactly that realisation: the interesting engineering is upstream of the model.

The ideas that carry

  • RAG is one of four options, not the default. Fine-tuning, long context and tool use solve overlapping problems. RAG earns its place when knowledge is large, changing, or must be cited.
  • Chunking is the highest-leverage decision you will make and the one most often made by accident. Chunk boundaries decide what can ever be retrieved together.
  • Dense and sparse retrieval fail differently, which is why hybrid wins. Embeddings capture meaning and miss exact identifiers; BM25 does the reverse. Fusion beats either alone on real corpora.
  • Reranking is where cheap wins live. Retrieve broadly with a fast index, then spend a cross-encoder on the top candidates. Precision at the top is what the generator actually sees.
  • Query transformation fixes the mismatch between how people ask and how documents are written. Rewriting, decomposition and HyDE all attack the same gap.
  • If you can't measure retrieval separately from generation, you can't debug either. Recall@k and nDCG on retrieval; faithfulness and answer relevance on generation. The triad exists so a bad answer can be attributed.
  • Grounded generation is a context-assembly problem. Ordering, deduplication, citation scaffolding and an explicit "say you don't know" path do more for trustworthiness than any prompt phrasing.
  • Access control has to live in the retrieval layer. A permission check after generation is already too late — the leak happened when the chunk entered the context.

What you can now do

Build RAG from scratch and know what each framework abstraction is hiding. Ship pgvector-backed retrieval. Evaluate a pipeline component by component. Decide, with evidence, whether a problem wants RAG, fine-tuning, long context or a tool.

If one thing sticks

When the answer is wrong, look at what was retrieved before you look at the prompt. It is almost always the retrieval.