832 citations · 1.7k across the 27 of their papers we have counts for
21 papers · 1 filter
Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
Ayush Tewari, Tianwei Yin, George Cazenavette +5
Denoising diffusion models are a powerful type of generative models used to capture complex distributions of real-world signals. However, their applicability is limited to scenario…
Materialistic: Selecting Similar Materials in Images
Prafull Sharma, Julien Philip, Michaël Gharbi +3
Separating an image into meaningful underlying components is a crucial first step for both editing and understanding images. We present a method capable of selecting the regions of…
FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
Guangxuan Xiao, Tianwei Yin, William T. Freeman +2
Diffusion models excel at text-to-image generation, especially in subject-driven generation for personalized images. However, existing methods are inefficient due to the subject-sp…
Can Shadows Reveal Biometric Information?
Safa C. Medin, Amir Weiss, Frédo Durand +2
We study the problem of extracting biometric information of individuals by looking at shadows of objects cast on diffuse surfaces. We show that the biometric information leakage fr…
Neural Groundplans: Persistent Neural Scene Representations from a Single Image
Prafull Sharma, Ayush Tewari, Yilun Du +7
We present a method to map 2D image observations of a scene to a persistent 3D scene representation, enabling novel view synthesis and disentangled representation of the movable an…
Unsupervised Discovery and Composition of Object Light Fields
Cameron Smith, Hong-Xing Yu, Sergey Zakharov +4
Neural scene representations, both continuous and discrete, have recently emerged as a powerful new paradigm for 3D scene understanding. Recent efforts have tackled unsupervised di…
Learning to generate line drawings that convey geometry and semantics
Caroline Chan, Fredo Durand, Phillip Isola
This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, t…
What You Can Learn by Staring at a Blank Wall
Prafull Sharma, Miika Aittala, Yoav Y. Schechner +4
We present a passive non-line-of-sight method that infers the number of people or activity of a person from the observation of a blank wall in an unknown room. Our technique analyz…
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Vincent Sitzmann, Semon Rezchikov, William T. Freeman +2
Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neur…
When and how CNNs generalize to out-of-distribution category-viewpoint combinations
Spandan Madan, Timothy Henry, Jamell Dozier +6
Object recognition and viewpoint estimation lie at the heart of visual understanding. Recent works suggest that convolutional neural networks (CNNs) fail to generalize to out-of-di…
Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization
Miika Aittala, Prafull Sharma, Lukas Murmann +4
We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by fact…
A Dataset of Multi-Illumination Images in the Wild
Lukas Murmann, Michael Gharbi, Miika Aittala +1
Collections of images under a single, uncontrolled illumination have enabled the rapid advancement of core computer vision tasks like classification, detection, and segmentation. B…
Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions
Guha Balakrishnan, Adrian V. Dalca, Amy Zhao +3
We introduce visual deprojection: the task of recovering an image or video that has been collapsed along a dimension. Projections arise in various contexts, such as long-exposure p…
Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation
Ronnachai Jaroensri, Camille Biscarrat, Miika Aittala +1
Image reconstruction techniques such as denoising often need to be applied to the RGB output of cameras and cellphones. Unfortunately, the commonly used additive white noise (AWGN)…
Data augmentation using learned transformations for one-shot medical image segmentation
Amy Zhao, Guha Balakrishnan, Frédo Durand +2
Image segmentation is an important task in many medical applications. Methods based on convolutional neural networks attain state-of-the-art accuracy; however, they typically rely…
Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics
Spandan Madan, Zoya Bylinskii, Matthew Tancik +6
Widely used in news, business, and educational media, infographics are handcrafted to effectively communicate messages about complex and often abstract topics including `ways to co…
Synthesizing Images of Humans in Unseen Poses
Guha Balakrishnan, Amy Zhao, Adrian V. Dalca +2
We address the computational problem of novel human pose synthesis. Given an image of a person and a desired pose, we produce a depiction of that person in that pose, retaining the…
Learning-based Video Motion Magnification
Tae-Hyun Oh, Ronnachai Jaroensri, Changil Kim +4
Video motion magnification techniques allow us to see small motions previously invisible to the naked eyes, such as those of vibrating airplane wings, or swaying buildings under th…
Understanding Infographics through Textual and Visual Tag Prediction
Zoya Bylinskii, Sami Alsheikh, Spandan Madan +5
We introduce the problem of visual hashtag discovery for infographics: extracting visual elements from an infographic that are diagnostic of its topic. Given an infographic as inpu…
A Video-Based Method for Objectively Rating Ataxia
Ronnachai Jaroensri, Amy Zhao, Guha Balakrishnan +4
For many movement disorders, such as Parkinson's disease and ataxia, disease progression is visually assessed by a clinician using a numerical disease rating scale. These tests are…
What do different evaluation metrics tell us about saliency models?
Zoya Bylinskii, Tilke Judd, Aude Oliva +2
How best to evaluate a saliency model's ability to predict where humans look in images is an open research question. The choice of evaluation metric depends on how saliency is defi…