Paper2014
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever, Oriol Vinyals, Quoc V. Le · NIPS · 2014
Abstract
Establishes the encoder-decoder pattern: read a whole input sequence into a fixed-length vector, then generate the output sequence from it, so input and output need not be the same length or even the same language. Reversing the source sequence produced a large improvement, which hinted at how badly long-range dependencies were being handled.
Why it matters
The structural ancestor of every generative model you use. Its weakness — squeezing a whole sentence through one vector — is the specific problem attention was invented to solve three years later.
architecturepretrainingtransfer
Read the source
https://arxiv.org/abs/1409.3215