2.3k citations · 6.3k across the 61 of their papers we have counts for
65 papers
Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO
Yangkun Chen, Joseph Suarez, Junjie Zhang +18
We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-a…
Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
William Shen, Ge Yang, Alan Yu +3
Self-supervised and language-supervised image models contain rich knowledge of the world that is important for generalization. Many robotic tasks, however, require a detailed under…
MultiEarth 2023 -- Multimodal Learning for Earth and Environment Workshop and Challenge
Miriam Cha, Gregory Angelides, Mark Hamilton +6
The Multimodal Learning for Earth and Environment Workshop (MultiEarth 2023) is the second annual CVPR workshop aimed at the monitoring and analysis of the health of Earth ecosyste…
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data
Stephanie Fu, Netanel Tamir, Shobhita Sundaram +4
Current perceptual similarity metrics operate at the level of pixels and patches. These metrics compare images in terms of their low-level colors and textures, but fail to capture…
StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
Yonglong Tian, Lijie Fan, Phillip Isola +2
We investigate the potential of learning visual representations using synthetic images generated by text-to-image models. This is a natural question in the light of the excellent p…
Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
Minyoung Huh, Brian Cheung, Pulkit Agrawal +1
This work examines the challenges of training neural networks using vector quantization using straight-through estimation. We find that a primary cause of training instability is t…
Improving CLIP Training with Language Rewrites
Lijie Fan, Dilip Krishnan, Phillip Isola +2
Contrastive Language-Image Pre-training (CLIP) stands as one of the most effective and scalable methods for training transferable vision models using paired image and text data. CL…
Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
Tongzhou Wang, Antonio Torralba, Phillip Isola +1
In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetric structure. This paper introduces Quasimetric Reinforcement Le…
Persistent Nature: A Generative Model of Unbounded 3D Worlds
Lucy Chai, Richard Tucker, Zhengqi Li +2
Despite increasingly realistic image quality, recent 3D image generative models often operate on 3D volumes of fixed extent with limited camera motions. We investigate the task of…
Steerable Equivariant Representation Learning
Sangnie Bhardwaj, Willie McClinton, Tongzhou Wang +4
Pre-trained deep image representations are useful for post-training tasks such as classification through transfer learning, image retrieval, and object detection. Data augmentation…
Procedural Image Programs for Representation Learning
Manel Baradad, Chun-Fu Chen, Jonas Wulff +4
Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias. Existing wor…
Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings
Tongzhou Wang, Phillip Isola
Asymmetrical distance structures (quasimetrics) are ubiquitous in our lives and are gaining more attention in machine learning applications. Imposing such quasimetric structures in…
Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
Kevin Frans, Phillip Isola
One of the grand challenges of reinforcement learning is the ability to generalize to new tasks. However, general agents require a set of rich, diverse tasks to train on. Designing…
Totems: Physical Objects for Verifying Visual Integrity
Jingwei Ma, Lucy Chai, Minyoung Huh +4
We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems…
Developing a Series of AI Challenges for the United States Department of the Air Force
Vijay Gadepally, Gregory Angelides, Andrei Barbu +39
Through a series of federal initiatives and orders, the U.S. Government has been making a concerted effort to ensure American leadership in AI. These broad strategy documents have…
Semantic uncertainty intervals for disentangled latent spaces
Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates +2
Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on t…
On the Learning and Learnability of Quasimetrics
Tongzhou Wang, Phillip Isola
Our world is full of asymmetries. Gravity and wind can make reaching a place easier than coming back. Social artifacts such as genealogy charts and citation graphs are inherently d…
Denoised MDPs: Learning World Models Better Than the World Itself
Tongzhou Wang, Simon S. Du, Antonio Torralba +3
The ability to separate signal from noise, and reason with clean abstractions, is critical to intelligence. With this ability, humans can efficiently perform real world tasks witho…
NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields
Lin Yen-Chen, Pete Florence, Jonathan T. Barron +3
Thin, reflective objects such as forks and whisks are common in our daily lives, but they are particularly challenging for robot perception because it is hard to reconstruct them u…
Any-resolution Training for High-resolution Image Synthesis
Lucy Chai, Michael Gharbi, Eli Shechtman +2
Generative models operate at fixed resolution, even though natural images come in a variety of sizes. As high-resolution details are downsampled away and low-resolution images are…
Learning to generate line drawings that convey geometry and semantics
Caroline Chan, Fredo Durand, Phillip Isola
This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, t…
Exploring Visual Prompts for Adapting Large-Scale Models
Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan +1
We investigate the efficacy of visual prompting to adapt large-scale models in vision. Following the recent approach from prompt tuning and adversarial reprogramming, we learn a si…
Contrastive Feature Loss for Image Prediction
Alex Andonian, Taesung Park, Bryan Russell +3
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result. Yet, this basic functionality remains an open problem. A popular…
Learning to Ground Multi-Agent Communication with Autoencoders
Toru Lin, Minyoung Huh, Chris Stauffer +2
Communication requires having a common language, a lingua franca, between agents. This language could emerge via a consensus process, but it may require many generations of trial a…