1k citations · 3.9k across the 28 of their papers we have counts for
4 papers · 1 filter
Scalable Diffusion Models with Transformers
William Peebles, Saining Xie
We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a tran…
Exploring Long-Sequence Masked Autoencoders
Ronghang Hu, Shoubhik Debnath, Saining Xie +1
Masked Autoencoding (MAE) has emerged as an effective approach for pre-training representations across multiple domains. In contrast to discrete tokens in natural languages, the in…
A ConvNet for the 2020s
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu +3
The "Roaring 20s" of visual recognition began with the introduction of Vision Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art image classification mo…
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision
Ajinkya Tejankar, Maziar Sanjabi, Bichen Wu +4
Using natural language as a supervision for training visual recognition models holds great promise. Recent works have shown that if such supervision is used in the form of alignmen…