Research Library
Library
Papers, patents & whitepapers, annotated
MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
Mingqiao Ye, Zhaochong An, Zhitong Gao et al. · ICML 2026 · 2026
The self-verification trick is the striking part: a model that can render its answer in another modality is able to grade itself. Accepted at ICML 2026.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Daya Guo, Dejian Yang et al. · Nature volume 645, pages 633-638 (2025) · 2025
The result that reframed reasoning as something trained rather than prompted. That the behaviour emerges from outcome-only rewards — nobody demonstrated the reasoning — is the part worth sitting with.
OLMo: Accelerating the Science of Language Models
Dirk Groeneveld, Iz Beltagy, Pete Walsh et al. · ACL · 2024
'Open-weights' and 'open' are not the same claim, and this is the paper that makes the distinction concrete. If you want to study how a model acquires a capability during training rather than just probe the finished artefact, this is the one that lets you.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafael Rafailov, Archit Sharma, Eric Mitchell et al. · NeurIPS · 2023
It made preference tuning something a small team can run, which is why most open fine-tunes now use this rather than the reinforcement-learning pipeline it replaced. A good example of a derivation, not a bigger model, being the contribution.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao, Daniel Y. Fu, Stefano Ermon et al. · NeurIPS · 2022
The rare optimisation with no accuracy trade-off to weigh, which is why it was absorbed everywhere within a year. It is also the clearest lesson that on modern hardware the bottleneck is usually memory movement, not compute.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al. · NeurIPS · 2022
The single most consequential result on this shelf for how AI products actually feel. It established that instruction-following is a separate axis from scale — and made the case that a smaller aligned model can beat a much larger unaligned one.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, Wang, Schuurmans et al. · NeurIPS · 2022
Why "let's think step by step" works. The practical root of every reasoning-heavy prompt and the reasoning-model wave that followed.
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis et al. · ICLR · 2021
The reason fine-tuning is something an individual can do on one GPU rather than a datacentre operation. Nearly every 'custom model' offering is this underneath.
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy et al. · ICML · 2021
Where zero-shot became a practical expectation rather than a research curiosity. The shared image-text space it introduced is still the substrate for multimodal search and for how most image generators are conditioned.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
William Fedus, Barret Zoph, Noam Shazeer · J. Mach. Learn. Res. · 2021
The architecture behind most frontier models today, where a very large parameter count coexists with a modest per-token cost. Read it before the Kimi and DeepSeek reports here — it is the idea they are both building on.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, Perez, Piktus et al. · NeurIPS · 2020
The origin of "RAG". Explains why bolting a retriever onto a model beats fine-tuning for fast-changing knowledge — the pattern behind most production AI search today.
Dense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin, Barlas Oğuz, Sewon Min et al. · EMNLP · 2020
The retrieval half of RAG, and the reason vector search became the default. Read it with the RAG paper here — this is the component that made that architecture work.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Adam Roberts et al. · J. Mach. Learn. Res. · 2019
Valuable as a controlled study rather than a model release: it is one of the few places where pretraining decisions are compared like-for-like instead of confounded. The text-to-text framing is now simply how everyone uses language models.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee et al. · NAACL-HLT · 2018
The paper that made 'pretrain once, fine-tune cheaply' the default. Encoder models like this still underpin most retrieval and classification stacks, including the reranking step in a typical RAG pipeline.
Attention Is All You Need
Vaswani, Shazeer, Parmar et al. · NeurIPS · 2017
The paper the entire modern LLM era is built on. If you read one source here, read this one — every model you use is a descendant of this architecture.
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · CVPR · 2015
Residual connections are in every transformer block you will ever use, which makes this required background even though the paper is about images. The framing is the lesson: the obstacle to depth was trainability, not capacity.
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · ICLR · 2014
Still the default optimiser for essentially every model on this shelf, more than a decade on. Worth reading precisely because it is the piece of the stack most people never look inside.
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le · NIPS · 2014
The structural ancestor of every generative model you use. Its weakness — squeezing a whole sentence through one vector — is the specific problem attention was invented to solve three years later.
Efficient Estimation of Word Representations in Vector Space
Tomas Mikolov, Kai Chen, Greg Corrado et al. · ICLR · 2013
Where the idea that meaning can live in a vector became practical, and the direct ancestor of every embedding in your retrieval stack. Also the most approachable paper here — the model is simple enough to read in one sitting.
Showing 1–19 of 19 sources · newest first
About Nybble™
The AI space moves fast.
Nybble™ is how you keep up — and stay sharp.
What happened. In two minutes.
The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.
Go to SnackThe concepts behind the headlines.
News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.
Go to StackProve you actually get it.
Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.
Go to Hack