Patent2021
Processing sequences using convolutional neural networks
DeepMind Technologies Ltd · USPTO — US11080591B2 · 2021
Abstract
Covers generating sequences with stacked dilated causal convolutions rather than recurrence — each layer skips over a widening gap so the receptive field grows quickly with depth, while causality ensures no output depends on future inputs. The claims describe the architecture and its application to sequence generation.
Why it matters
The convolutional answer to sequence modelling, filed as the transformer was arriving. A good illustration that the architecture that wins is not the only one that worked.
patentarchitectureefficiency
Read the source
https://patents.google.com/patent/US11080591B2/en