2.3k citations · 6.3k across the 61 of their papers we have counts for
40 papers · 1 filter
Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
William Shen, Ge Yang, Alan Yu +3
Self-supervised and language-supervised image models contain rich knowledge of the world that is important for generalization. Many robotic tasks, however, require a detailed under…
MultiEarth 2023 -- Multimodal Learning for Earth and Environment Workshop and Challenge
Miriam Cha, Gregory Angelides, Mark Hamilton +6
The Multimodal Learning for Earth and Environment Workshop (MultiEarth 2023) is the second annual CVPR workshop aimed at the monitoring and analysis of the health of Earth ecosyste…
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data
Stephanie Fu, Netanel Tamir, Shobhita Sundaram +4
Current perceptual similarity metrics operate at the level of pixels and patches. These metrics compare images in terms of their low-level colors and textures, but fail to capture…
StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
Yonglong Tian, Lijie Fan, Phillip Isola +2
We investigate the potential of learning visual representations using synthetic images generated by text-to-image models. This is a natural question in the light of the excellent p…
Improving CLIP Training with Language Rewrites
Lijie Fan, Dilip Krishnan, Phillip Isola +2
Contrastive Language-Image Pre-training (CLIP) stands as one of the most effective and scalable methods for training transferable vision models using paired image and text data. CL…
Persistent Nature: A Generative Model of Unbounded 3D Worlds
Lucy Chai, Richard Tucker, Zhengqi Li +2
Despite increasingly realistic image quality, recent 3D image generative models often operate on 3D volumes of fixed extent with limited camera motions. We investigate the task of…
Steerable Equivariant Representation Learning
Sangnie Bhardwaj, Willie McClinton, Tongzhou Wang +4
Pre-trained deep image representations are useful for post-training tasks such as classification through transfer learning, image retrieval, and object detection. Data augmentation…
Procedural Image Programs for Representation Learning
Manel Baradad, Chun-Fu Chen, Jonas Wulff +4
Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias. Existing wor…
Totems: Physical Objects for Verifying Visual Integrity
Jingwei Ma, Lucy Chai, Minyoung Huh +4
We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems…
Semantic uncertainty intervals for disentangled latent spaces
Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates +2
Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on t…
Any-resolution Training for High-resolution Image Synthesis
Lucy Chai, Michael Gharbi, Eli Shechtman +2
Generative models operate at fixed resolution, even though natural images come in a variety of sizes. As high-resolution details are downsampled away and low-resolution images are…
Learning to generate line drawings that convey geometry and semantics
Caroline Chan, Fredo Durand, Phillip Isola
This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, t…
Exploring Visual Prompts for Adapting Large-Scale Models
Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan +1
We investigate the efficacy of visual prompting to adapt large-scale models in vision. Following the recent approach from prompt tuning and adversarial reprogramming, we learn a si…
Contrastive Feature Loss for Image Prediction
Alex Andonian, Taesung Park, Bryan Russell +3
Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result. Yet, this basic functionality remains an open problem. A popular…
Learning to See by Looking at Noise
Manel Baradad, Jonas Wulff, Tongzhou Wang +2
Current vision systems are trained on huge datasets, and these datasets come with costs: curation is expensive, they inherit human biases, and there are concerns over privacy and u…
Generative Models as a Data Source for Multiview Representation Learning
Ali Jahanian, Xavier Puig, Yonglong Tian +1
Generative models are now capable of producing highly realistic images that look nearly indistinguishable from the data on which they are trained. This raises the question: if we h…
Curious Representation Learning for Embodied Intelligence
Yilun Du, Chuang Gan, Phillip Isola
Self-supervised representation learning has achieved remarkable success in recent years. By subverting the need for supervised labels, such approaches are able to utilize the numer…
Ensembling with Deep Generative Views
Lucy Chai, Jun-Yan Zhu, Eli Shechtman +2
Recent generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose, simply by learning from unlabeled image col…
Explaining in Style: Training a GAN to explain a classifier in StyleSpace
Oran Lang, Yossi Gandelsman, Michal Yarom +8
Image classification models can depend on multiple different semantic attributes of the image. An explanation of the decision of the classifier needs to both discover and visualize…
Using latent space regression to analyze and leverage compositionality in GANs
Lucy Chai, Jonas Wulff, Phillip Isola
In recent years, Generative Adversarial Networks have become ubiquitous in both research and public perception, but how GANs convert an unstructured latent code to a high quality o…
INeRF: Inverting Neural Radiance Fields for Pose Estimation
Lin Yen-Chen, Pete Florence, Jonathan T. Barron +3
We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF). NeRFs have been shown to be remarkably effective for the task of…
What makes fake images detectable? Understanding properties that generalize
Lucy Chai, David Bau, Ser-Nam Lim +1
The quality of image generation and manipulation is reaching impressive levels, making it increasingly difficult for a human to distinguish between what is real and what is fake. H…
Noisy Agents: Self-supervised Exploration by Predicting Auditory Events
Chuang Gan, Xiaoyu Chen, Phillip Isola +2
Humans integrate multiple sensory modalities (e.g. visual and audio) to build a causal understanding of the physical world. In this work, we propose a novel type of intrinsic motiv…
What Makes for Good Views for Contrastive Learning?
Yonglong Tian, Chen Sun, Ben Poole +3
Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning. Despite its succ…