DeepView: View Synthesis with Learned Gradient Descent

Abstract

We present a novel approach to view synthesis using multiplane images (MPIs). Building on recent advances in learned gradient descent, our algorithm generates an MPI from a set of sparse camera viewpoints. The resulting method incorporates occlusion reasoning, improving performance on challenging scene features such as object boundaries, lighting reflections, thin structures, and scenes with high depth complexity. We show that our method achieves high-quality, state-of-the-art results on two datasets: the Kalantari light field dataset, and a new camera array dataset, Spaces, which we make publicly available.

See https://augmentedperception.github.io/deepview/ for more results, video and an interactive viewer

References in corpus (3)

• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Training Deep Nets with Sublinear Memory Cost
• DeepStereo: Learning to Predict New Views from the World's Imagery

Cited by in corpus (8)

• Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework
• LiveView: Dynamic Target-Centered MPI for View Synthesis
• Open Challenges on Generating Referring Expressions for Human-Robot Interaction
• Embedding Novel Views in a Single JPEG Image
• Deep Feedback Inverse Problem Solver
• Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis
• Point-Based Neural Rendering with Per-View Optimization
• Human Body Model Fitting by Learned Gradient Descent