206 citations · 470 across the 23 of their papers we have counts for
10 papers · 1 filter
Seeing Voices: Generating A-Roll Video from Audio with Mirage
Aditi Sundararaman, Amogh Adishesha, Andrew Jaegle +10
From professional filmmaking to user-generated content, creators and consumers have long recognized that the power of video depends on the harmonious integration of what we hear (t…
SODA: Bottleneck Diffusion Models for Representation Learning
Drew A. Hudson, Daniel Zoran, Mateusz Malinowski +6
We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact rep…
Where Should I Spend My FLOPS? Efficiency Evaluations of Visual Pre-training Methods
Skanda Koppula, Yazhe Li, Evan Shelhamer +5
Self-supervised methods have achieved remarkable success in transfer learning, often achieving the same or better accuracy than supervised pre-training. Most prior work has done so…
Transframer: Arbitrary Frame Prediction with Generative Models
Charlie Nash, João Carreira, Jacob Walker +4
We present a general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. Our approach unifies a broad range of tasks, from image segment…
Object discovery and representation networks
Olivier J. Hénaff, Skanda Koppula, Evan Shelhamer +5
The promise of self-supervised learning (SSL) is to leverage large amounts of unlabeled data to solve complex tasks. While there has been excellent progress with simple, image-leve…
HiP: Hierarchical Perceiver
Joao Carreira, Skanda Koppula, Daniel Zoran +10
General perception systems such as Perceivers can process arbitrary modalities in any combination and are able to handle up to a few hundred thousand inputs. They achieve this gene…
Perceiver: General Perception with Iterative Attention
Andrew Jaegle, Felix Gimeno, Andrew Brock +3
Biological systems perceive the world by simultaneously processing high-dimensional inputs from modalities as diverse as vision, audition, touch, proprioception, etc. The perceptio…
Predicting the Future with Transformational States
Andrew Jaegle, Oleh Rybkin, Konstantinos G. Derpanis +1
An intelligent observer looks at the world and sees not only what is, but what is moving and what can be moved. In other words, the observer sees how the present state of the world…
Understanding image motion with group representations
Andrew Jaegle, Stephen Phillips, Daphne Ippolito +1
Motion is an important signal for agents in dynamic environments, but learning to represent motion from unlabeled video is a difficult and underconstrained problem. We propose a mo…
Fast, Robust, Continuous Monocular Egomotion Computation
Andrew Jaegle, Stephen Phillips, Kostas Daniilidis
We propose robust methods for estimating camera egomotion in noisy, real-world monocular image sequences in the general case of unknown observer rotation and translation with two v…