832 citations · 1.7k across the 27 of their papers we have counts for
34 papers
Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
Ayush Tewari, Tianwei Yin, George Cazenavette +5
Denoising diffusion models are a powerful type of generative models used to capture complex distributions of real-world signals. However, their applicability is limited to scenario…
Materialistic: Selecting Similar Materials in Images
Prafull Sharma, Julien Philip, Michaël Gharbi +3
Separating an image into meaningful underlying components is a crucial first step for both editing and understanding images. We present a method capable of selecting the regions of…
FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
Guangxuan Xiao, Tianwei Yin, William T. Freeman +2
Diffusion models excel at text-to-image generation, especially in subject-driven generation for personalized images. However, existing methods are inefficient due to the subject-sp…
Can Shadows Reveal Biometric Information?
Safa C. Medin, Amir Weiss, Frédo Durand +2
We study the problem of extracting biometric information of individuals by looking at shadows of objects cast on diffuse surfaces. We show that the biometric information leakage fr…
Gemino: Practical and Robust Neural Compression for Video Conferencing
Vibhaalakshmi Sivaraman, Pantea Karimi, Vedantha Venkatapathy +5
Video conferencing systems suffer from poor user experience when network conditions deteriorate because current video codecs simply cannot operate at extremely low bitrates. Recent…
Neural Groundplans: Persistent Neural Scene Representations from a Single Image
Prafull Sharma, Ayush Tewari, Yilun Du +7
We present a method to map 2D image observations of a scene to a persistent 3D scene representation, enabling novel view synthesis and disentangled representation of the movable an…
Differentiable Rendering of Neural SDFs through Reparameterization
Sai Praveen Bangaru, Michaël Gharbi, Tzu-Mao Li +7
We present a method to automatically compute correct gradients with respect to geometric scene parameters in neural SDF renderers. Recent physically-based differentiable rendering…
Unsupervised Discovery and Composition of Object Light Fields
Cameron Smith, Hong-Xing Yu, Sergey Zakharov +4
Neural scene representations, both continuous and discrete, have recently emerged as a powerful new paradigm for 3D scene understanding. Recent efforts have tackled unsupervised di…
Learning to generate line drawings that convey geometry and semantics
Caroline Chan, Fredo Durand, Phillip Isola
This paper presents an unpaired method for creating line drawings from photographs. Current methods often rely on high quality paired datasets to generate line drawings. However, t…
What You Can Learn by Staring at a Blank Wall
Prafull Sharma, Miika Aittala, Yoav Y. Schechner +4
We present a passive non-line-of-sight method that infers the number of people or activity of a person from the observation of a blank wall in an unknown room. Our technique analyz…
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Vincent Sitzmann, Semon Rezchikov, William T. Freeman +2
Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neur…
Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
Xin Yuan, Yang Liu, Jinli Suo +2
We consider the reconstruction problem of video snapshot compressive imaging (SCI), which captures high-speed videos using a low-speed 2D sensor (detector). The underlying principl…
AsyncTaichi: On-the-fly Inter-kernel Optimizations for Imperative and Spatially Sparse Programming
Yuanming Hu, Mingkuan Xu, Ye Kuang +1
Leveraging spatial sparsity has become a popular approach to accelerate 3D computer graphics applications. Spatially sparse data structures and efficient sparse kernels (such as pa…
When and how CNNs generalize to out-of-distribution category-viewpoint combinations
Spandan Madan, Timothy Henry, Jamell Dozier +6
Object recognition and viewpoint estimation lie at the heart of visual understanding. Recent works suggest that convolutional neural networks (CNNs) fail to generalize to out-of-di…
Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings
Amy Zhao, Guha Balakrishnan, Kathleen M. Lewis +3
We introduce a new video synthesis task: synthesizing time lapse videos depicting how a given painting might have been created. Artists paint using unique combinations of brushes,…
Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization
Miika Aittala, Prafull Sharma, Lukas Murmann +4
We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by fact…
A Dataset of Multi-Illumination Images in the Wild
Lukas Murmann, Michael Gharbi, Miika Aittala +1
Collections of images under a single, uncontrolled illumination have enabled the rapid advancement of core computer vision tasks like classification, detection, and segmentation. B…
DiffTaichi: Differentiable Programming for Physical Simulation
Yuanming Hu, Luke Anderson, Tzu-Mao Li +4
We present DiffTaichi, a new differentiable programming language tailored for building high-performance differentiable physical simulators. Based on an imperative programming langu…
Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions
Guha Balakrishnan, Adrian V. Dalca, Amy Zhao +3
We introduce visual deprojection: the task of recovering an image or video that has been collapsed along a dimension. Projections arise in various contexts, such as long-exposure p…
Flexible SVBRDF Capture with a Multi-Image Deep Network
Valentin Deschaintre, Miika Aittala, Fredo Durand +2
Empowered by deep learning, recent methods for material capture can estimate a spatially-varying reflectance from a single photograph. Such lightweight capture is in stark contrast…
Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation
Ronnachai Jaroensri, Camille Biscarrat, Miika Aittala +1
Image reconstruction techniques such as denoising often need to be applied to the RGB output of cameras and cellphones. Unfortunately, the commonly used additive white noise (AWGN)…
Data augmentation using learned transformations for one-shot medical image segmentation
Amy Zhao, Guha Balakrishnan, Frédo Durand +2
Image segmentation is an important task in many medical applications. Methods based on convolutional neural networks attain state-of-the-art accuracy; however, they typically rely…
Single-Image SVBRDF Capture with a Rendering-Aware Deep Network
Valentin Deschaintre, Miika Aittala, Fredo Durand +2
Texture, highlights, and shading are some of many visual cues that allow humans to perceive material appearance in single pictures. Yet, recovering spatially-varying bi-directional…
FFT Convolutions are Faster than Winograd on Modern CPUs, Here is Why
Aleksandar Zlateski, Zhen Jia, Kai Li +1
Winograd-based convolution has quickly gained traction as a preferred approach to implement convolutional neural networks (ConvNet) on various hardware platforms because it require…