110 citations · 435 across the 31 of their papers we have counts for
36 papers
A Deep Perceptual Measure for Lens and Camera Calibration
Yannick Hold-Geoffroy, Dominique Piché-Meunier, Kalyan Sunkavalli +3
Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrica…
Physically-Based Editing of Indoor Scene Lighting from a Single Image
Zhengqin Li, Jia Shi, Sai Bi +6
We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. This is an extremely challenging problem that…
NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction
Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli +2
While NeRF has shown great success for neural reconstruction and rendering, its limited MLP capacity and long per-scene optimization times make it challenging to model large-scale…
NeuTex: Neural Texture Mapping for Volumetric Neural Rendering
Fanbo Xiang, Zexiang Xu, Miloš Hašan +3
Recent work has demonstrated that volumetric scene representations combined with differentiable volume rendering can enable photo-realistic rendering for challenging scenes that me…
Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments
Zhihao Xia, Michaël Gharbi, Federico Perazzi +2
We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments. Our goal is to produce a high-…
MaterialGAN: Reflectance Capture using a Generative SVBRDF Model
Yu Guo, Cameron Smith, Miloš Hašan +2
We address the problem of reconstructing spatially-varying BRDFs from a small set of image measurements. This is a fundamentally under-constrained problem, and previous work has re…
Neural Reflectance Fields for Appearance Acquisition
Sai Bi, Zexiang Xu, Pratul Srinivasan +6
We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene using a fully-conn…
Deep Multi Depth Panoramas for View Synthesis
Kai-En Lin, Zexiang Xu, Ben Mildenhall +6
We propose a learning-based approach for novel view synthesis for multi-camera 360 panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing…
Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli +4
We present a deep learning approach to reconstruct scene appearance from unstructured images captured under collocated point lighting. At the heart of Deep Reflectance Volumes is a…
OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
Zhengqin Li, Ting-Wei Yu, Shen Sang +14
We propose a novel framework for creating large-scale photorealistic datasets of indoor scenes, with ground truth geometry, material, lighting and semantics. Our goal is to make th…
Single View Metrology in the Wild
Rui Zhu, Xingyi Yang, Yannick Hold-Geoffroy +4
Most 3D reconstruction methods may only recover scene properties up to a global scale ambiguity. We present a novel approach to single view metrology that can recover the absolute…
State of the Art on Neural Rendering
Ayush Tewari, Ohad Fried, Justus Thies +16
Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic imag…
Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement
Sai Bi, Kalyan Sunkavalli, Federico Perazzi +3
We present a method to improve the visual realism of low-quality, synthetic images, e.g. OpenGL renderings. Training an unpaired synthetic-to-real translation network in image spac…
Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli +2
We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object from a sparse set of only six images…
Automatic Scene Inference for 3D Object Compositing
Kevin Karsch, Kalyan Sunkavalli, Sunil Hadap +4
We present a user-friendly image editing system that supports a drag-and-drop object insertion (where the user merely drags objects into the image, and the system automatically pla…
Basis Prediction Networks for Effective Burst Denoising with Large Kernels
Zhihao Xia, Federico Perazzi, Michaël Gharbi +2
Bursts of images exhibit significant self-similarity across both time and space. This motivates a representation of the kernels as linear combinations of a small set of basis eleme…
Deep Parametric Indoor Lighting Estimation
Marc-André Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli +2
We present a method to estimate lighting from a single image of an indoor scene. Previous work has used an environment map representation that does not account for the localized na…
All-Weather Deep Outdoor Lighting Estimation
Jinsong Zhang, Kalyan Sunkavalli, Yannick Hold-Geoffroy +3
We present a neural network that predicts HDR outdoor illumination from a single LDR image. At the heart of our work is a method to accurately learn HDR lighting from LDR panoramas…
Fast Spatially-Varying Indoor Lighting Estimation
Mathieu Garon, Kalyan Sunkavalli, Sunil Hadap +2
We propose a real-time method to estimate spatiallyvarying indoor lighting from a single RGB image. Given an image and a 2D location in that image, our CNN estimates a 5th order sp…
Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi +2
We propose a deep inverse rendering framework for indoor scenes. From a single RGB image of an arbitrary indoor scene, we create a complete scene reconstruction, estimating shape,…
Learning to Separate Multiple Illuminants in a Single Image
Zhuo Hui, Ayan Chakrabarti, Kalyan Sunkavalli +1
We present a method to separate a single image captured under two illuminants, with different spectra, into the two images corresponding to the appearance of the scene under each i…
MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Xinchen Yan, Akash Rastogi, Ruben Villegas +5
Long-term human motion can be represented as a series of motion modes---motion sequences that capture short-term temporal dynamics---with transitions between them. We leverage this…
Self-supervised Multi-view Person Association and Its Applications
Minh Vo, Ersin Yumer, Kalyan Sunkavalli +3
Reliable markerless motion tracking of people participating in a complex group activity from multiple moving cameras is challenging due to frequent occlusions, strong viewpoint and…
Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image
Zhengqin Li, Kalyan Sunkavalli, Manmohan Chandraker
We propose a material acquisition approach to recover the spatially-varying BRDF and normal map of a near-planar surface from a single image captured by a handheld mobile phone cam…