CM3: A Causal Masked Multimodal Model of the Internet

Armen Aghajanyan, Bernie Huang, Candace Ross +8

We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens. Our new…