1.4k citations · 1.7k across the 11 of their papers we have counts for
13 papers
LiT: Zero-Shot Transfer with Locked-image text Tuning
Xiaohua Zhai, Xiao Wang, Basil Mustafa +4
This paper presents contrastive-tuning, a simple method employing contrastive training to align image and text models while still taking advantage of their pre-training. In our emp…
The Impact of Reinitialization on Generalization in Convolutional Neural Networks
Ibrahim Alabdulmohsin, Hartmut Maennel, Daniel Keysers
Recent results suggest that reinitializing a subset of the parameters of a neural network during training can improve generalization, particularly for small training sets. We study…
Continental-Scale Building Detection from High Resolution Satellite Imagery
Wojciech Sirko, Sergii Kashubin, Marvin Ritter +7
Identifying the locations and footprints of buildings is vital for many practical and scientific purposes. Such information can be particularly useful in developing regions where a…
A Generalized Lottery Ticket Hypothesis
Ibrahim Alabdulmohsin, Larisa Markeeva, Daniel Keysers +1
We introduce a generalization to the lottery ticket hypothesis in which the notion of "sparsity" is relaxed by choosing an arbitrary basis in the space of parameters. We present ev…
MLP-Mixer: An all-MLP Architecture for Vision
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov +9
Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this…
Scaling Vision with Sparse Mixture of Experts
Carlos Riquelme, Joan Puigcerver, Basil Mustafa +5
Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated excellent scalability in Natural Language Processing. In Computer Vision, however, almost all performant network…