22k citations · 27.6k across the 41 of their papers we have counts for
6 papers · 1 filter
FlexiViT: One Model for All Patch Sizes
Lucas Beyer, Pavel Izmailov, Alexander Kolesnikov +7
Vision Transformers convert images to sequences by slicing them into patches. The size of these patches controls a speed/accuracy tradeoff, with smaller patches leading to higher a…
VeLO: Training Versatile Learned Optimizers by Scaling Up
Luke Metz, James Harrison, C. Daniel Freeman +8
While deep learning models have replaced hand-designed features across many domains, these models are still trained with hand-designed optimizers. In this work, we leverage the sam…
PaLI: A Jointly-Scaled Multilingual Language-Image Model
Xi Chen, Xiao Wang, Soravit Changpinyo +26
Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this…
Better plain ViT baselines for ImageNet-1k
Lucas Beyer, Xiaohua Zhai, Alexander Kolesnikov
It is commonly accepted that the Vision Transformer model requires sophisticated regularization techniques to excel at ImageNet-1k scale data. Surprisingly, we find this is not the…
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes
Alexander Kolesnikov, André Susano Pinto, Lucas Beyer +3
We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks. In contrast to previous models, UViM has the same functional form for all tasks; it…
Kubric: A scalable dataset generator
Klaus Greff, Francois Belletti, Lucas Beyer +32
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and trainin…