Paper2021
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al. · ICML · 2021
Abstract
Trains an image encoder and a text encoder together on a very large set of image-caption pairs, so that matching pairs land near each other in a shared space. Because classification becomes a matter of comparing an image to descriptions written in plain language, the model classifies categories it was never explicitly trained on.
Why it matters
Where zero-shot became a practical expectation rather than a research curiosity. The shared image-text space it introduced is still the substrate for multimodal search and for how most image generators are conditioned.
multimodalpretrainingtransferevals
Read the source
https://arxiv.org/abs/2103.00020