219 citations · 400 across the 13 of their papers we have counts for
15 papers
Learning Complex Non-Rigid Image Edits from Multimodal Conditioning
Nikolai Warner, Jack Kolb, Meera Hahn +3
In this paper we focus on inserting a given human (specifically, a single image of a person) into a novel scene. Our method, which builds on top of Stable Diffusion, yields natural…
Sample what you cant compress
Vighnesh Birodkar, Gabriel Barcik, James Lyon +3
For learned image representations, basic autoencoders often produce blurry results. Reconstruction quality can be improved by incorporating additional penalties such as adversarial…
VideoPoet: A Large Language Model for Zero-Shot Video Generation
Dan Kondratyuk, Lijun Yu, Xiuye Gu +28
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals. VideoPoet employs a decoder-on…
Text and Click inputs for unambiguous open vocabulary instance segmentation
Nikolai Warner, Meera Hahn, Jonathan Huang +2
Segmentation localizes objects in an image on a fine-grained per-pixel scale. Segmentation benefits by humans-in-the-loop to provide additional input of objects to segment using a…
Scaling Vision Transformers to 22 Billion Parameters
Mostafa Dehghani, Josip Djolonga, Basil Mustafa +39
The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Visio…
Open-Vocabulary Temporal Action Detection with Off-the-Shelf Image-Text Features
Vivek Rathod, Bryan Seybold, Sudheendra Vijayanarasimhan +4
Detecting actions in untrimmed videos should not be limited to a small, closed set of classes. We present a simple, yet effective strategy for open-vocabulary temporal action detec…
Proper Reuse of Image Classification Features Improves Object Detection
Cristina Vasconcelos, Vighnesh Birodkar, Vincent Dumoulin
A common practice in transfer learning is to initialize the downstream model weights by pre-training on a data-abundant upstream task. In object detection specifically, the feature…
Less is More: Generating Grounded Navigation Instructions from Landmarks
Su Wang, Ceslee Montgomery, Jordi Orbay +7
We study the automatic generation of navigation instructions from 360-degree images captured on indoor routes. Existing generators suffer from poor visual grounding, causing them t…
The iWildCam 2021 Competition Dataset
Sara Beery, Arushi Agarwal, Elijah Cole +1
Camera traps enable the automatic collection of large quantities of image data. Ecologists use camera traps to monitor animal populations all over the world. In order to estimate t…
The surprising impact of mask-head architecture on novel class segmentation
Vighnesh Birodkar, Zhichao Lu, Siyang Li +2
Instance segmentation models today are very accurate when trained on large annotated datasets, but collecting mask annotations at scale is prohibitively expensive. We address the p…
A Closed-Form Learned Pooling for Deep Classification Networks
Vighnesh Birodkar, Hossein Mobahi, Dilip Krishnan +1
In modern computer vision tasks, convolutional neural networks (CNNs) are indispensable for image classification tasks due to their efficiency and effectiveness. Part of their supe…
Straight to the point: reinforcement learning for user guidance in ultrasound
Fausto Milletari, Vighnesh Birodkar, Michal Sofka
Point of care ultrasound (POCUS) consists in the use of ultrasound imaging in critical or emergency situations to support clinical decisions by healthcare professionals and first r…
Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Need
Vighnesh Birodkar, Hossein Mobahi, Samy Bengio
Large datasets have been crucial to the success of deep learning models in the recent years, which keep performing better as they are trained with more labelled data. While there h…
Unsupervised Learning of Disentangled Representations from Video
Remi Denton, Vighnesh Birodkar
We present a new model DrNET that learns disentangled image representations from video. Our approach leverages the temporal coherence of video and a novel adversarial loss to learn…
A convolutional approach to reflection symmetry
Marcelo Cicconet, Vighnesh Birodkar, Mads Lund +2
We present a convolutional approach to reflection symmetry detection in 2D. Our model, built on the products of complex-valued wavelet convolutions, simplifies previous edge-based…