353 citations · 492 across the 2 of their papers we have counts for
4 papers · 1 filter
Early Convolutions Help Transformers See Better
Tete Xiao, Mannat Singh, Eric Mintun +3
Vision transformer (ViT) models exhibit substandard optimizability. In particular, they are sensitive to the choice of optimizer (AdamW vs. SGD), optimizer hyperparameters, and tra…
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
Aishwarya Kamath, Mannat Singh, Yann LeCun +3
Multi-modal reasoning systems rely on a pre-trained object detector to extract regions of interest from the image. However, this crucial module is typically used as a black box, tr…
Fast and Accurate Model Scaling
Piotr Dollár, Mannat Singh, Ross Girshick
In this work we analyze strategies for convolutional neural network scaling; that is, the process of scaling a base convolutional network to endow it with greater computational com…
Self-supervised Pretraining of Visual Features in the Wild
Priya Goyal, Mathilde Caron, Benjamin Lefaudeux +8
Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and SwAV have reduced the gap with supervised methods. These results have been achieved in a control environment,…