4.4k citations · 6.8k across the 29 of their papers we have counts for
33 papers
4M: Massively Multimodal Masked Modeling
David Mizrahi, Roman Bachmann, Oğuzhan Fatih Kar +4
Current machine learning models for vision are often highly specialized and limited to a single modality and task. In contrast, recent large language models exhibit a wide range of…
Unraveling the Key Components of OOD Generalization via Diversification
Harold Benoit, Liangze Jiang, Andrei Atanov +3
Supervised learning datasets may contain multiple cues that explain the training set equally well, i.e., learning any of them would lead to the correct predictions on the training…
Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedback
Teresa Yeo, Oğuzhan Fatih Kar, Zahra Sodagar +1
We propose a method for adapting neural networks to distribution shifts at test-time. In contrast to training-time robustness mechanisms that attempt to anticipate and counter the…
Modality-invariant Visual Odometry for Embodied Vision
Marius Memmel, Roman Bachmann, Amir Zamir
Effectively localizing an agent in a realistic, noisy setting is crucial for many embodied vision tasks. Visual Odometry (VO) is a practical substitute for unreliable GPS and compa…
An Information-Theoretic Approach to Transferability in Task Transfer Learning
Yajie Bao, Yang Li, Shao-Lun Huang +4
Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks. An important question i…
PALMER: Perception-Action Loop with Memory for Long-Horizon Planning
Onur Beker, Mohammad Mohammadi, Amir Zamir
To achieve autonomy in a priori unknown real-world scenarios, agents should be able to: i) act from high-dimensional sensory observations (e.g., images), ii) learn from past experi…
Task Discovery: Finding the Tasks that Neural Networks Generalize on
Andrei Atanov, Andrei Filatov, Teresa Yeo +2
When developing deep learning models, we usually decide what task we want to solve then search for a model that generalizes well on the task. An intriguing question would be: what…
3D Common Corruptions and Data Augmentation
Oğuzhan Fatih Kar, Teresa Yeo, Andrei Atanov +1
We introduce a set of image transformations that can be used as corruptions to evaluate the robustness of models as well as data augmentation mechanisms for training neural network…
MultiMAE: Multi-modal Multi-task Masked Autoencoders
Roman Bachmann, David Mizrahi, Andrei Atanov +1
We propose a pre-training strategy called Multi-modal Multi-task Masked Autoencoders (MultiMAE). It differs from standard Masked Autoencoding in two key aspects: I) it can optional…
Simple Control Baselines for Evaluating Transfer Learning
Andrei Atanov, Shijian Xu, Onur Beker +2
Transfer learning has witnessed remarkable progress in recent years, for example, with the introduction of augmentation-based contrastive self-supervised learning methods. While a…
CLIPasso: Semantically-Aware Object Sketching
Yael Vinker, Ehsan Pajouheshgar, Jessica Y. Bo +5
Abstraction is at the heart of sketching due to the simple and minimal nature of line drawings. Abstraction entails identifying the essential visual properties of an object or scen…
Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans
Ainaz Eftekhar, Alexander Sax, Roman Bachmann +2
This paper introduces a pipeline to parametrically sample and render multi-task vision datasets from comprehensive 3D scans from the real world. Changing the sampling parameters al…
Robustness via Cross-Domain Ensembles
Teresa Yeo, Oğuzhan Fatih Kar, Alexander Sax +1
We present a method for making neural network predictions robust to shifts from the training data distribution. The proposed method is based on making predictions via a diverse set…
Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
Bryan Chen, Alexander Sax, Gene Lewis +5
Vision-based robotics often separates the control loop into one module for perception and a separate module for control. It is possible to train the whole system end-to-end (e.g. w…
Robust Learning Through Cross-Task Consistency
Amir Zamir, Alexander Sax, Teresa Yeo +6
Visual perception entails solving a wide set of tasks, e.g., object detection, depth estimation, etc. The predictions made for multiple tasks from the same image are not independen…
Learning to Navigate Using Mid-Level Visual Priors
Alexander Sax, Jeffrey O. Zhang, Bradley Emi +4
How much does having visual priors about the world (e.g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e.g. navigating a complex environment)…
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Jeffrey O Zhang, Alexander Sax, Amir Zamir +2
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights. Adaptation can be useful in…
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera
Iro Armeni, Zhi-Yang He, JunYoung Gwak +4
A comprehensive semantic understanding of a scene is important for many applications - but in what space should diverse semantic information (e.g., objects, scene categories, mater…
Which Tasks Should Be Learned Together in Multi-task Learning?
Trevor Standley, Amir R. Zamir, Dawn Chen +3
Many computer vision applications require solving multiple tasks in real-time. A neural network can be trained to solve multiple tasks simultaneously using multi-task learning. Thi…
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
Alexander Sax, Bradley Emi, Amir R. Zamir +3
How much does having visual priors about the world (e.g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e.g. delivering a package)? We study t…
Gibson Env: Real-World Perception for Embodied Agents
Fei Xia, Amir Zamir, Zhi-Yang He +3
Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently l…
On Evaluation of Embodied Navigation Agents
Peter Anderson, Angel Chang, Devendra Singh Chaplot +8
Skillful mobile operation in three-dimensional environments is a primary topic of study in Artificial Intelligence. The past two years have seen a surge of creative work on navigat…
Taskonomy: Disentangling Task Transfer Learning
Amir Zamir, Alexander Sax, William Shen +3
Do visual tasks have a relationship, or are they unrelated? For instance, could having surface normals simplify estimating the depth of an image? Intuition answers these questions…
Generic 3D Representation via Pose Estimation and Matching
Amir R. Zamir, Tilman Wekel, Pulkit Argrawal +3
Though a large body of computer vision research has investigated developing generic semantic representations, efforts towards developing a similar representation for 3D has been li…