18.1k citations · 35.5k across the 59 of their papers we have counts for
72 papers
Segment Anything
Alexander Kirillov, Eric Mintun, Nikhila Ravi +9
We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest seg…
The effectiveness of MAE pre-pretraining for billion-scale pretraining
Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala +9
This paper revisits the standard pretrain-then-finetune paradigm used in computer vision for visual recognition tasks. Typically, state-of-the-art foundation models are pretrained…
Exploring Plain Vision Transformer Backbones for Object Detection
Yanghao Li, Hanzi Mao, Ross Girshick +1
We explore the plain, non-hierarchical Vision Transformer (ViT) as a backbone network for object detection. This design enables the original ViT architecture to be fine-tuned for o…
Revisiting Weakly Supervised Pre-Training of Visual Perception Models
Mannat Singh, Laura Gustafson, Aaron Adcock +7
Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent st…
Masked Autoencoders Are Scalable Vision Learners
Kaiming He, Xinlei Chen, Saining Xie +3
This paper shows that masked autoencoders (MAE) are scalable self-supervised learners for computer vision. Our MAE approach is simple: we mask random patches of the input image and…
Benchmarking Detection Transfer Learning with Vision Transformers
Yanghao Li, Saining Xie, Xinlei Chen +3
Object detection is a central downstream task used to test if pre-trained network parameters confer benefits, such as improved accuracy or training speed. The complexity of object…
PyTorchVideo: A Deep Learning Library for Video Understanding
Haoqi Fan, Tullie Murrell, Heng Wang +13
We introduce PyTorchVideo, an open-source deep-learning library that provides a rich set of modular, efficient, and reproducible components for a variety of video understanding tas…
Early Convolutions Help Transformers See Better
Tete Xiao, Mannat Singh, Eric Mintun +3
Vision transformer (ViT) models exhibit substandard optimizability. In particular, they are sensitive to the choice of optimizer (AdamW vs. SGD), optimizer hyperparameters, and tra…
A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning
Christoph Feichtenhofer, Haoqi Fan, Bo Xiong +2
We present a large-scale study on unsupervised spatiotemporal representation learning from videos. With a unified perspective on four recent image-based frameworks, we study a simp…
Boundary IoU: Improving Object-Centric Image Segmentation Evaluation
Bowen Cheng, Ross Girshick, Piotr Dollár +2
We present Boundary IoU (Intersection-over-Union), a new segmentation evaluation measure focused on boundary quality. We perform an extensive analysis across different error types…
Fast and Accurate Model Scaling
Piotr Dollár, Mannat Singh, Ross Girshick
In this work we analyze strategies for convolutional neural network scaling; that is, the process of scaling a base convolutional network to endow it with greater computational com…
Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details
Achal Dave, Piotr Dollár, Deva Ramanan +2
By design, average precision (AP) for object detection aims to treat all classes independently: AP is computed independently per category and averaged. On one hand, this is desirab…
Large scale weakly and semi-supervised learning for low-resource video ASR
Kritika Singh, Vimal Manohar, Alex Xiao +7
Many semi- and weakly-supervised approaches have been investigated for overcoming the labeling cost of building high quality speech recognition systems. On the challenging task of…
Designing Network Design Spaces
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick +2
In this work, we present a new network design paradigm. Our goal is to help advance the understanding of network design and discover design principles that generalize across settin…
Improved Baselines with Momentum Contrastive Learning
Xinlei Chen, Haoqi Fan, Ross Girshick +1
Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR'…
Are Labels Necessary for Neural Architecture Search?
Chenxi Liu, Piotr Dollár, Kaiming He +3
Existing neural network architectures in computer vision -- whether designed by humans or by machines -- were typically found using both images and their associated labels. In this…
PointRend: Image Segmentation as Rendering
Alexander Kirillov, Yuxin Wu, Kaiming He +1
We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over- a…
A Multigrid Method for Efficiently Training Video Models
Chao-Yuan Wu, Ross Girshick, Kaiming He +2
Training competitive deep video models is an order of magnitude slower than training their counterpart image models. Slow training causes long research cycles, which hinders progre…
Momentum Contrast for Unsupervised Visual Representation Learning
Kaiming He, Haoqi Fan, Yuxin Wu +2
We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary…
Training ASR models by Generation of Contextual Information
Kritika Singh, Dmytro Okhonko, Jun Liu +8
Supervised ASR models have reached unprecedented levels of accuracy, thanks in part to ever-increasing amounts of labelled training data. However, in many applications and locales,…
LVIS: A Dataset for Large Vocabulary Instance Segmentation
Agrim Gupta, Piotr Dollár, Ross Girshick
Progress on object detection is enabled by datasets that focus the research community's attention on open challenges. This process led us from simple images to complex scenes and f…
PHYRE: A New Benchmark for Physical Reasoning
Anton Bakhtin, Laurens van der Maaten, Justin Johnson +2
Understanding and reasoning about physics is an important ability of intelligent agents. We develop the PHYRE benchmark for physical reasoning that contains a set of simple classic…
Exploring Randomly Wired Neural Networks for Image Recognition
Saining Xie, Alexander Kirillov, Ross Girshick +1
Neural networks for image recognition have evolved through extensive manual design from simple chain-like models to structures with multiple wiring paths. The success of ResNets an…
TensorMask: A Foundation for Dense Object Segmentation
Xinlei Chen, Ross Girshick, Kaiming He +1
Sliding-window object detectors that generate bounding-box object predictions over a dense, regular grid have advanced rapidly and proven popular. In contrast, modern instance segm…