A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality

Abstract

We present a system for acquiring, processing, and rendering panoramic light field still photography for display in Virtual Reality (VR). We acquire spherical light field datasets with two novel light field camera rigs designed for portable and efficient light field acquisition. We introduce a novel real-time light field reconstruction algorithm that uses a per-view geometry and a disk-based blending field. We also demonstrate how to use a light field prefiltering operation to project from a high-quality offline reconstruction model into our real-time model while suppressing artifacts. We introduce a practical approach for compressing light fields by modifying the VP9 video codec to provide high quality compression with real-time, random access decompression. We combine these components into a complete light field system offering convenient acquisition, compact file size, and high-quality rendering while generating stereo views at 90Hz on commodity VR hardware. Using our system, we built a freely available light field experience application called Welcome to Light Fields featuring a library of panoramic light field stills for consumer VR which has been downloaded over 15,000 times.

arXiv comment15 pages, 14 figures, 2 tables, accepted by SIGGRAPH Asia 2018, low-resolution version

References in corpus (1)

• Learning-Based View Synthesis for Light Field Cameras

Cited by in corpus (14)

• Neural Volumes: Learning Dynamic Renderable Volumes from Images
• Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
• Spatial-Angular Attention Network for Light Field Reconstruction
• Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network
• VR-NeRF: High-Fidelity Virtualized Walkable Spaces
• Light Field View Synthesis via Aperture Disparity and Warping Confidence Map
• 3D Scene Geometry Estimation from 360${}^{\circ }$ Imagery: A Survey
• Learning Occlusion-Aware View Synthesis for Light Fields
• TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis
• LapEPI-Net: A Laplacian Pyramid EPI structure for Learning-based Dense Light Field Reconstruction
• SILVR: A Synthetic Immersive Large-Volume Plenoptic Dataset
• Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency
• Deep Coarse-to-fine Dense Light Field Reconstruction with Flexible Sampling and Geometry-aware Fusion
• Hallucinated Neural Radiance Fields in the Wild