22k citations · 34.8k across the 45 of their papers we have counts for
28 papers · 1 filter
Moving Off-the-Grid: Scene-Grounded Video Representations
Sjoerd van Steenkiste, Daniel Zoran, Yi Yang +13
Current vision models typically maintain a fixed correspondence between their representation structure and image space. Each layer comprises a set of tokens arranged "on-the-grid,"…
ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization
Luca Eyring, Shyamgopal Karthik, Karsten Roth +2
Text-to-Image (T2I) models have made significant advancements in recent years, but they still struggle to accurately capture intricate details specified in complex compositional pr…
Simple Open-Vocabulary Object Detection with Vision Transformers
Matthias Minderer, Alexey Gritsenko, Austin Stone +11
Combining simple architectures with large-scale pre-training has led to massive improvements in image classification. For object detection, pre-training and scaling approaches are…
Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations
Mehdi S. M. Sajjadi, Henning Meyer, Etienne Pot +10
A classical problem in computer vision is to infer a 3D scene representation from few images that can be used to render novel views at interactive rates. Previous work focuses on r…
Conditional Object-Centric Learning from Video
Thomas Kipf, Gamaleldin F. Elsayed, Aravindh Mahendran +6
Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built. Rec…
Do Vision Transformers See Like Convolutional Neural Networks?
Maithra Raghu, Thomas Unterthiner, Simon Kornblith +2
Convolutional neural networks (CNNs) have so far been the de-facto model for visual data. Recent work has shown that (Vision) Transformer models (ViT) can achieve comparable or eve…
MLP-Mixer: An all-MLP Architecture for Vision
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov +9
Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this…
Differentiable Patch Selection for Image Recognition
Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy +3
Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand…
Learning Object-Centric Video Models by Contrasting Sets
Sindy Löwe, Klaus Greff, Rico Jonschkowski +2
Contrastive, self-supervised learning of object representations recently emerged as an attractive alternative to reconstruction-based training. Prior approaches focus on contrastin…
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov +9
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention…
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi +3
We present a learning-based method for synthesizing novel views of complex scenes using only unstructured collections of in-the-wild photographs. We build on Neural Radiance Fields…
Learning Depth With Very Sparse Supervision
Antonio Loquercio, Alexey Dosovitskiy, Davide Scaramuzza
Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D pr…
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov +14
Representation learning promises to unlock deep learning for the long tail of vision tasks without expensive labelled datasets. Yet, the absence of a unified evaluation for general…
Benchmarking Classic and Learned Navigation in Complex 3D Environments
Dmytro Mishkin, Alexey Dosovitskiy, Vladlen Koltun
Navigation research is attracting renewed interest with the advent of learning-based methods. However, this new line of work is largely disconnected from well-established classic n…
Unsupervised Learning of Shape and Pose with Differentiable Point Clouds
Eldar Insafutdinov, Alexey Dosovitskiy
We address the problem of learning accurate 3D shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the s…
On Offline Evaluation of Vision-based Driving Models
Felipe Codevilla, Antonio M. López, Vladlen Koltun +1
Autonomous driving models should ideally be evaluated by deploying them on a fleet of physical vehicles in the real world. Unfortunately, this approach is not practical for the vas…
What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?
Nikolaus Mayer, Eddy Ilg, Philipp Fischer +4
The finding that very large networks can be trained efficiently and reliably has led to a paradigm shift in computer vision from engineered solutions to learning formulations. As a…
Artistic style transfer for videos and spherical images
Manuel Ruder, Alexey Dosovitskiy, Thomas Brox
Manually re-drawing an image in a certain artistic style takes a professional artist a long time. Doing this for a video sequence single-handedly is beyond imagination. We present…
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Maxim Tatarchenko, Alexey Dosovitskiy, Thomas Brox
We present a deep convolutional decoder architecture that can generate volumetric 3D outputs in a compute- and memory-efficient manner by using an octree representation. The networ…
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia +3
The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined…
DeMoN: Depth and Motion Network for Learning Monocular Stereo
Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig +4
In this paper we formulate structure from motion as a learning problem. We train a convolutional network end-to-end to compute depth and camera motion from successive, unconstraine…
Artistic style transfer for videos
Manuel Ruder, Alexey Dosovitskiy, Thomas Brox
In the past, manually re-drawing an image in a certain artistic style required a professional artist and a long time. Doing this for a video sequence single-handed was beyond imagi…
A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
Nikolaus Mayer, Eddy Ilg, Philip Häusser +4
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-ca…
Multi-view 3D Models from Single Images with a Convolutional Network
Maxim Tatarchenko, Alexey Dosovitskiy, Thomas Brox
We present a convolutional network capable of inferring a 3D representation of a previously unseen object given a single image of this object. Concretely, the network can predict a…