Research Library
Library
Papers, patents & whitepapers, annotated
GPT-4 Technical Report
OpenAI, Josh Achiam, Steven Adler et al. · OpenAI (Technical Report) · 2023
The moment frontier reports stopped being reproducible science and became capability-and-safety disclosures. Worth reading alongside an open report like DeepSeek's or OLMo's to see exactly which questions each one refuses to answer.
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu et al. · Anthropic (Technical Report) · 2022
Notable for making the values explicit and auditable rather than implicit in whichever labels the annotators happened to produce. The self-critique loop is also the ancestor of the LLM-as-judge pipelines now used far beyond safety work.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao et al. · BigScience (Technical Report) · 2022
The first serious demonstration that a frontier-scale model could be built outside a private lab. Its language coverage remains unusual — most open models are still overwhelmingly English.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao, Daniel Y. Fu, Stefano Ermon et al. · NeurIPS · 2022
The rare optimisation with no accuracy trade-off to weigh, which is why it was absorbed everywhere within a year. It is also the clearest lesson that on modern hardware the bottleneck is usually memory movement, not compute.
Training Compute-Optimal Large Language Models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch et al. · arXiv · 2022
The result that redirected the field from 'bigger' to 'better fed', and the reason the models that followed got smaller and cheaper to serve while getting better. If you read one scaling paper, read this one.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al. · NeurIPS · 2022
The single most consequential result on this shelf for how AI products actually feel. It established that instruction-following is a separate axis from scale — and made the case that a smaller aligned model can beat a much larger unaligned one.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, Wang, Schuurmans et al. · NeurIPS · 2022
Why "let's think step by step" works. The practical root of every reasoning-heavy prompt and the reasoning-model wave that followed.
Processing sequences using convolutional neural networks
DeepMind Technologies Ltd · USPTO — US11080591B2 · 2021
The convolutional answer to sequence modelling, filed as the transformer was arriving. A good illustration that the architecture that wins is not the only one that worked.
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis et al. · ICLR · 2021
The reason fine-tuning is something an individual can do on one GPU rather than a datacentre operation. Nearly every 'custom model' offering is this underneath.
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy et al. · ICML · 2021
Where zero-shot became a practical expectation rather than a research curiosity. The shared image-text space it introduced is still the substrate for multimodal search and for how most image generators are conditioned.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
William Fedus, Barret Zoph, Noam Shazeer · J. Mach. Learn. Res. · 2021
The architecture behind most frontier models today, where a very large parameter count coexists with a modest per-token cost. Read it before the Kimi and DeepSeek reports here — it is the idea they are both building on.
Language Models are Few-Shot Learners
Brown, Mann, Ryder et al. · arXiv · 2020
The GPT-3 paper — where "just give it a few examples in the prompt" became a real strategy. It reframed prompting as programming.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, Perez, Piktus et al. · NeurIPS · 2020
The origin of "RAG". Explains why bolting a retriever onto a model beats fine-tuning for fast-changing knowledge — the pattern behind most production AI search today.
Multi-task learning using knowledge distillation
Google LLC · USPTO — US10635977B2 · 2020
Distillation as claimed intellectual property, and the technique behind nearly every small model that punches above its size — including several on this shelf. Read it next to the on-policy distillation work here to see how far the idea has since travelled.
Dense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin, Barlas Oğuz, Sewon Min et al. · EMNLP · 2020
The retrieval half of RAG, and the reason vector search became the default. Read it with the RAG paper here — this is the component that made that architecture work.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Adam Roberts et al. · J. Mach. Learn. Res. · 2019
Valuable as a controlled study rather than a model release: it is one of the few places where pretraining decisions are compared like-for-like instead of confounded. The text-to-text framing is now simply how everyone uses language models.
Attention-based sequence transduction neural networks
Google LLC · USPTO — US10452978B2 · 2019
The Transformer, as a granted patent. Useful for seeing how a landmark architecture is framed in IP terms — and a reminder that the research and the patent often diverge.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee et al. · NAACL-HLT · 2018
The paper that made 'pretrain once, fine-tune cheaply' the default. Encoder models like this still underpin most retrieval and classification stacks, including the reranking step in a typical RAG pipeline.
Attention Is All You Need
Vaswani, Shazeer, Parmar et al. · NeurIPS · 2017
The paper the entire modern LLM era is built on. If you read one source here, read this one — every model you use is a descendant of this architecture.
System and method for addressing overfitting in a neural network
Google LLC · USPTO — US9406017B2 · 2016
Dropout, patented. Worth a look because it is one of the clearest cases of a technique that spread through the field as a free idea while also existing as an enforceable claim.
Showing 41–60 of 66 sources · newest first
About Nybble™
The AI space moves fast.
Nybble™ is how you keep up — and stay sharp.
What happened. In two minutes.
The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.
Go to SnackThe concepts behind the headlines.
News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.
Go to StackProve you actually get it.
Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.
Go to Hack