1.2k citations · 2.8k across the 42 of their papers we have counts for
34 papers · 1 filter
A Theory of Topological Derivatives for Inverse Rendering of Geometry
Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi
We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives for variational optimizat…
NeRFs: The Search for the Best 3D Representation
Ravi Ramamoorthi
Neural Radiance Fields or NeRFs have become the representation of choice for problems in view synthesis or image-based rendering, as well as in many other applications across compu…
PVP: Personalized Video Prior for Editable Dynamic Portraits using StyleGAN
Kai-En Lin, Alex Trevithick, Keli Cheng +5
Portrait synthesis creates realistic digital avatars which enable users to interact with others in a compelling way. Recent advances in StyleGAN and its extensions have shown promi…
Real-Time Radiance Fields for Single-Image Portrait View Synthesis
Alex Trevithick, Matthew Chan, Michael Stengel +7
We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our im…
Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation
Liwen Wu, Rui Zhu, Mustafa B. Yaldiz +7
Inverse path tracing has recently been applied to joint material and lighting estimation, given geometry and multi-view HDR observations of an indoor scene. However, it has two maj…
NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion
Jiatao Gu, Alex Trevithick, Kai-En Lin +4
Novel view synthesis from a single image requires inferring occluded regions of objects and scenes whilst simultaneously maintaining semantic and physical consistency with the inpu…
Vision Transformer for NeRF-Based View Synthesis from a Single Input Image
Kai-En Lin, Lin Yen-Chen, Wei-Sheng Lai +3
Although neural radiance fields (NeRF) have shown impressive advances for novel view synthesis, most methods typically require multiple input images of the same scene with accurate…
Physically-Based Editing of Indoor Scene Lighting from a Single Image
Zhengqin Li, Jia Shi, Sai Bi +6
We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. This is an extremely challenging problem that…
A Level Set Theory for Neural Implicit Evolution under Explicit Flows
Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi
Coordinate-based neural networks parameterizing implicit surfaces have emerged as efficient representations of geometry. They effectively act as parametric level sets with the zero…
Learning Neural Transmittance for Efficient Rendering of Reflectance Fields
Mohammad Shafiei, Sai Bi, Zhengqin Li +3
Recently neural volumetric representations such as neural reflectance fields have been widely applied to faithfully reproduce the appearance of real-world objects and scenes under…
View Synthesis of Dynamic Scenes based on Deep 3D Mask Volume
Kai-En Lin, Guowei Yang, Lei Xiao +2
Image view synthesis has seen great success in reconstructing photorealistic visuals, thanks to deep learning and various novel representations. The next key step in immersive virt…
NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting
Tiancheng Sun, Kai-En Lin, Sai Bi +2
Human portraits exhibit various appearances when observed from different views under different lighting conditions. We can easily imagine how the face will look like in another set…
Modulated Periodic Activations for Generalizable Local Functional Representations
Ishit Mehta, Michaël Gharbi, Connelly Barnes +3
Multi-Layer Perceptrons (MLPs) make powerful functional representations for sampling and reconstruction problems involving low-dimensional signals like images,shapes and light fiel…
Real-Time Selfie Video Stabilization
Jiyang Yu, Ravi Ramamoorthi, Keli Cheng +2
We propose a novel real-time selfie video stabilization method. Our method is completely automatic and runs at 26 fps. We use a 1D linear convolutional network to directly infer th…
Neural Reflectance Fields for Appearance Acquisition
Sai Bi, Zexiang Xu, Pratul Srinivasan +6
We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene using a fully-conn…
Deep Multi Depth Panoramas for View Synthesis
Kai-En Lin, Zexiang Xu, Ben Mildenhall +6
We propose a learning-based approach for novel view synthesis for multi-camera 360 panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing…
Neural Light Transport for Relighting and View Synthesis
Xiuming Zhang, Sean Fanello, Yun-Ta Tsai +10
The light transport (LT) of a scene describes how it appears under different lighting and viewing directions, and complete knowledge of a scene's LT enables the synthesis of novel…
Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli +4
We present a deep learning approach to reconstruct scene appearance from unstructured images captured under collocated point lighting. At the heart of Deep Reflectance Volumes is a…
OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
Zhengqin Li, Ting-Wei Yu, Shen Sang +14
We propose a novel framework for creating large-scale photorealistic datasets of indoor scenes, with ground truth geometry, material, lighting and semantics. Our goal is to make th…
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall +6
We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions in low-dimensional problem domain…
Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement
Sai Bi, Kalyan Sunkavalli, Federico Perazzi +3
We present a method to improve the visual realism of low-quality, synthetic images, e.g. OpenGL renderings. Training an unpaired synthetic-to-real translation network in image spac…
Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images
Sai Bi, Zexiang Xu, Kalyan Sunkavalli +2
We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object from a sparse set of only six images…
Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness
Shuo Cheng, Zexiang Xu, Shilin Zhu +4
We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D reconstruction from multiple RGB images. Multi-view stereo (MVS) aims to reconstruct fine-grained scene geomet…
Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi +2
We propose a deep inverse rendering framework for indoor scenes. From a single RGB image of an arbitrary indoor scene, we create a complete scene reconstruction, estimating shape,…