Research Library
Library
Papers, patents & whitepapers, annotated
Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
Siyuan Huang, Pengyu Cheng, Haotian Liu et al. · arXiv · 2026
Self-play is how systems surpassed human data in games, and this is a serious attempt to port that to general model capability. Read it alongside the other co-evolution work here to see the pattern forming.
Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problems
Gaurav Dadhich · arXiv · 2026
Read this before you build another retrieval layer for your agent. The quadratic cost argument alone reframes context management as an architecture decision rather than a tuning detail.
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
Junsong Chen, Jincheng Yu, Yitong Li et al. · arXiv · 2026
The clearest recent evidence that linear attention works as one half of a hybrid rather than as a wholesale replacement — the same trade-off now shaping long-context language models.
K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs
Hao Liang, Qihan Lin, Zhaoyang Han et al. · arXiv · 2026
A good demonstration that a benchmark score answers only the question the benchmark asks. Models that ace exam items still fumble the structure underneath them — a real gap for anyone building tutoring products.
Sample-Efficient Learning from Agent Experience
Chenhui Gou, Haoqin Tu, Yunhao Fang et al. · arXiv · 2026
The gap between 65 and 4 percent is the entire paper. How you train on collected experience matters far more than whether you collect it in the first place.
Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
Jian Hu, Huiying Li, Hao Zhang et al. · arXiv · 2026
Worth reading as an argument about research velocity rather than as a tool announcement: when every algorithm change has to thread through layers of framework glue, the framework has become the bottleneck.
Interactive Training 2: Auditable Control Plane for Live Model Training
Wentao Zhang, Xuanhe Pan, Han Zhou et al. · arXiv · 2026
The auditability is the point. Once agents can steer training runs, a written record of who changed what and what happened next stops being a nicety and becomes the only way to debug.
Multi-Turn On-Policy Distillation with Prefix Replay
Baohao Liao, Hanze Dong, Christof Monz et al. · arXiv · 2026
Pairs naturally with the relay-distillation work in this collection — both attack the same tension from opposite sides. Read them together if you are training agents on teacher supervision.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Daya Guo, Dejian Yang et al. · Nature volume 645, pages 633-638 (2025) · 2025
The result that reframed reasoning as something trained rather than prompted. That the behaviour emerges from outcome-only rewards — nobody demonstrated the reasoning — is the part worth sitting with.
DeepSeek-V3 Technical Report
DeepSeek-AI, Aixin Liu, Bei Feng et al. · DeepSeek (Technical Report) · 2024
Read it for the training economics rather than the benchmarks. It is the clearest public account of how a frontier-scale run is actually made affordable, and much of what followed in open-weight modelling traces back to choices documented here.
Qwen2.5 Technical Report
Qwen, :, An Yang et al. · Alibaba Qwen (Technical Report) · 2024
The most practical open-weight family to build on, because a consistent recipe across sizes means you can prototype small and scale up without changing behaviour underneath you.
The Llama 3 Herd of Models
Meta AI · Meta (Technical Report) · 2024
A rare open look at how a frontier-scale model is actually built end to end. Read it when you want the engineering reality behind the benchmarks.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla et al. · Microsoft (Technical Report) · 2024
The strongest evidence that data curation substitutes for parameters at the small end. Relevant if you are deciding between a hosted frontier model and something you can run on-device.
StarCoder 2 and The Stack v2: The Next Generation
Anton Lozhkov, Raymond Li, Loubna Ben Allal et al. · BigCode (Technical Report) · 2024
Notable for treating training-data provenance as a design problem with an actual mechanism rather than a disclaimer. If you care where code models get their material, this is the reference implementation.
OLMo: Accelerating the Science of Language Models
Dirk Groeneveld, Iz Beltagy, Pete Walsh et al. · ACL · 2024
'Open-weights' and 'open' are not the same claim, and this is the paper that makes the distinction concrete. If you want to study how a model acquires a capability during training rather than just probe the finished artefact, this is the one that lets you.
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Rohan Anil, Sebastian Borgeaud et al. · Google DeepMind (Technical Report) · 2023
The clearest statement of the native-multimodal bet: that joint training beats bolting encoders onto a language model. Worth pairing with the scaling study in this collection, which examines whether that bet pays off under a fixed compute budget.
Mistral 7B
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch et al. · Mistral AI (Technical Report) · 2023
The paper that made small open models credible. Both attention tricks are now near-universal, so it doubles as the clearest short explanation of why modern models serve cheaply.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin Stone et al. · Meta (Technical Report) · 2023
For years this was the reference text on how a chat model is actually aligned, because it showed the process rather than just the result. The two-reward-model split is the detail most worth carrying away.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafael Rafailov, Archit Sharma, Eric Mitchell et al. · NeurIPS · 2023
It made preference tuning something a small team can run, which is why most open fine-tunes now use this rather than the reinforcement-learning pipeline it replaced. A good example of a derivation, not a bigger model, being the contribution.
PaLM 2 Technical Report
Rohan Anil, Andrew M. Dai, Orhan Firat et al. · Google (Technical Report) · 2023
A useful marker of when the industry stopped equating scale with parameters. Read it next to the Chinchilla paper here — this is that argument applied at production scale.
Showing 21–40 of 66 sources · newest first
About Nybble™
The AI space moves fast.
Nybble™ is how you keep up — and stay sharp.
What happened. In two minutes.
The AI news cycle moves at a pace no one can keep up with. Snack distills what launched, what shipped, and what matters — every day, without the filler.
Go to SnackThe concepts behind the headlines.
News tells you what. Stack tells you why and how. From RAG architectures to agentic evals, these are the ideas that will shape what you build next.
Go to StackProve you actually get it.
Reading about LangChain is not the same as knowing it. Hack challenges you with production-grade questions, then shows you the references that make the answer stick.
Go to Hack