Whitepaper2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao, Angela Fan et al. · BigScience (Technical Report) · 2022
Abstract
A 176-billion-parameter multilingual model produced by a year-long collaboration of hundreds of researchers on public supercomputing time, covering forty-six natural and thirteen programming languages. The report documents the governance and data-sourcing process as fully as the architecture, including how language communities were involved in assembling their own corpora.
Why it matters
The first serious demonstration that a frontier-scale model could be built outside a private lab. Its language coverage remains unusual — most open models are still overwhelmingly English.
open weightsmultilingualpretrainingopen source
Read the source
https://arxiv.org/abs/2211.05100