Nybble™
All courses

Guided Course

shipcodearchitecture

RAG Evaluation

Which stage failed? A metric per stage, ground truth that survives a re-chunk, and the ablation that tells you where to spend.

0/26
Start course
26 lessons
  1. 1. Why RAG Failures Are Hard to Attribute5 min
  2. 2. The Five Questions, and the Metric That Answers Each5 min
  3. 3. The Eval Set Is the Product5 min
  4. 4. Gold Chunks, and the Labelling Problem5 min
  5. 5. Query Taxonomy: What Kinds of Questions Break What4 min
  6. 6. Two Recalls: Set Membership and Statement Attribution4 min
  7. 7. Precision, MRR, NDCG, and Weighted Cumulative Precision5 min
  8. 8. Choosing k, and the Cost of Being Generous4 min
  9. 9. Contextual Relevancy: the Noise Problem4 min
  10. 10. Evaluating the Reranker in Isolation4 min
  11. 11. Faithfulness: Claims, Truths, and the Ambiguity Knob4 min
  12. 12. Faithfulness Is Not Hallucination: Supported ≠ True4 min
  13. 13. Answer Relevancy, and the Statement-Padding Effect3 min
  14. 14. Completeness and the Multi-Hop Trap4 min
  15. 15. Citation and Attribution Verification4 min
  16. 16. The Judge Under the Metrics4 min
  17. 17. Synthetic Test Set Generation, and Its Distribution Problem3 min
  18. 18. Why Every Metric Ships a `flaky` Flag3 min
  19. 19. Error Bars, and How Many Questions You Actually Need4 min
  20. 20. Component Attribution, Done Properly4 min
  21. 21. Evaluating the Techniques from Advanced RAG5 min
  22. 22. A New Stage: ASR, OCR, and Error Propagation4 min
  23. 23. When a Chunk Is a Time Span4 min
  24. 24. Four Paradigms, One Protocol4 min
  25. 25. Offline to Online, and Regression Gates5 min
  26. 26. Capstone — Evaluating a RAG System End to End4 min
  27. Key takeaways4 min
  28. How to get certified3 min
  29. Your certificate

About Nybble™

The AI space moves fast.
Nybble™ is how you keep up — and stay sharp.

Snack

What happened. In two minutes.

The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.

Go to Snack
Stack

The concepts behind the headlines.

News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.

Go to Stack
Hack

Prove you actually get it.

Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.

Go to Hack