Inverse Transport Networks

Abstract

We introduce inverse transport networks as a learning architecture for inverse rendering problems where, given input image measurements, we seek to infer physical scene parameters such as shape, material, and illumination. During training, these networks are evaluated not only in terms of how close they can predict groundtruth parameters, but also in terms of whether the parameters they produce can be used, together with physically-accurate graphics renderers, to reproduce the input image measurements. To en- able training of inverse transport networks using stochastic gradient descent, we additionally create a general-purpose, physically-accurate differentiable renderer, which can be used to estimate derivatives of images with respect to arbitrary physical scene parameters. Our experiments demonstrate that inverse transport networks can be trained efficiently using differentiable rendering, and that they generalize to scenes with completely unseen geometry and illumination better than networks trained without appearance- matching regularization.

References in corpus (7)

• Auto-Encoding Variational Bayes
• ADADELTA: An Adaptive Learning Rate Method
• Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks
• The Stan Math Library: Reverse-Mode Automatic Differentiation in C++
• Deep Lambertian Networks
• Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression
• LIME: Live Intrinsic Material Estimation

Cited by in corpus (9)

• High Resolution, Deep Imaging Using Confocal Time-of-flight Diffuse Optical Tomography
• Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer
• Blending Diverse Physical Priors with Neural Networks
• Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
• Deep Shape from Polarization
• Learning to Generate Synthetic 3D Training Data through Hybrid Gradient
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
• Visual Tomography: Physically Faithful Volumetric Models of Partially Translucent Objects
• AlgoNet: $C^{\infty }$ Smooth Algorithmic Neural Networks