Whitepaper2024
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah et al. · Microsoft (Technical Report) · 2024
Abstract
Microsoft's report on a small model family whose central claim is about data rather than scale: a heavily filtered corpus of web text plus synthetically generated material, curated for educational value, lets a few-billion-parameter model reach quality previously associated with much larger ones — small enough to run on a phone.
Why it matters
The strongest evidence that data curation substitutes for parameters at the small end. Relevant if you are deciding between a hosted frontier model and something you can run on-device.
small modelspretrainingefficiencytraining
Read the source
https://arxiv.org/abs/2404.14219