MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
Mingqiao Ye, Zhaochong An, Zhitong Gao, Xian Liu et al. · ICML 2026 · 2026
Abstract
Any-to-any models predict any modality from any combination of the others. They are normally built from scratch on encoder-decoder or diffusion architectures, which rules out starting from a strong pretrained decoder-only model. This work builds one on a decoder-only backbone treating every modality symmetrically, with no per-modality heads or losses — so the same model can chain generation through an intermediate modality, or check its own answer by generating a second one.
Why it matters
The self-verification trick is the striking part: a model that can render its answer in another modality is able to grade itself. Accepted at ICML 2026.
https://arxiv.org/abs/2607.25948