Whitepaper2023
Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert et al. · Meta (Technical Report) · 2023
Abstract
Meta's report on a family of open-weight models from seven to seventy billion parameters, and the first at that scale to document the whole alignment pipeline in detail: supervised fine-tuning, iterative reward modelling with separate helpfulness and safety models, and rejection sampling alongside reinforcement learning.
Why it matters
For years this was the reference text on how a chat model is actually aligned, because it showed the process rather than just the result. The two-reward-model split is the detail most worth carrying away.
open weightsfine tuningsafetytraining
Read the source
https://arxiv.org/abs/2307.09288