Paper2013
Efficient Estimation of Word Representations in Vector Space
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean · ICLR · 2013
Abstract
Two deliberately shallow models for learning word vectors from raw text — one predicting a word from its neighbours, the other the reverse. Dropping the hidden layer that earlier neural language models used made training fast enough to run on billions of words, and the resulting space turned out to encode relationships that could be manipulated arithmetically.
Why it matters
Where the idea that meaning can live in a vector became practical, and the direct ancestor of every embedding in your retrieval stack. Also the most approachable paper here — the model is simple enough to read in one sitting.
architecturepretrainingretrieval
Read the source
https://arxiv.org/abs/1301.3781