595 citations · 1.1k across the 5 of their papers we have counts for
8 papers
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
Mostafa Dehghani, Basil Mustafa, Josip Djolonga +12
The ubiquitous and demonstrably suboptimal choice of resizing images to a fixed resolution before processing them with computer vision models has not yet been successfully challeng…
Scaling Vision Transformers to 22 Billion Parameters
Mostafa Dehghani, Josip Djolonga, Basil Mustafa +39
The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Visio…
MetNet: A Neural Weather Model for Precipitation Forecasting
Casper Kaae Sønderby, Lasse Espeholt, Jonathan Heek +6
Weather forecasting is a long standing scientific challenge with direct social and economic impact. The task is suitable for deep neural networks due to vast amounts of continuousl…
Milking CowMask for Semi-Supervised Image Classification
Geoff French, Avital Oliver, Tim Salimans
Consistency regularization is a technique for semi-supervised learning that underlies a number of strong results for classification with few labeled data. It works by encouraging a…
S4L: Self-Supervised Semi-Supervised Learning
Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov +1
This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancin…
MixMatch: A Holistic Approach to Semi-Supervised Learning
David Berthelot, Nicholas Carlini, Ian Goodfellow +3
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current d…
When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets
Hong-Yu Zhou, Avital Oliver, Jianxin Wu +1
Semi-Supervised Learning (SSL) has been proved to be an effective way to leverage both labeled and unlabeled data at the same time. Recent semi-supervised approaches focus on deep…
Teacher-Student Curriculum Learning
Tambet Matiisen, Avital Oliver, Taco Cohen +1
We propose Teacher-Student Curriculum Learning (TSCL), a framework for automatic curriculum learning, where the Student tries to learn a complex task and the Teacher automatically…