Learning Intrinsic Image Decomposition from Watching the World

Abstract

Single-view intrinsic image decomposition is a highly ill-posed problem, and so a promising approach is to learn from large amounts of data. However, it is difficult to collect ground truth training data at scale for intrinsic images. In this paper, we explore a different approach to learning intrinsic images: observing image sequences over time depicting the same scene under changing illumination, and learning single-view decompositions that are consistent with these changes. This approach allows us to learn without ground truth decompositions, and to instead exploit information available from multiple images when training. Our trained model can then be applied at test time to single views. We describe a new learning framework based on this idea, including new loss functions that can be efficiently evaluated over entire sequences. While prior learning-based methods achieve good performance on specific benchmarks, we show that our approach generalizes well to several diverse datasets, including MIT intrinsic images, Intrinsic Images in the Wild and Shading Annotations in the Wild.

arXiv commentCVPR, 2018

References in corpus (3)

• Adam: A Method for Stochastic Optimization
• Self-Supervised Intrinsic Image Decomposition
• Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression

Cited by in corpus (10)

• Implicit Generation and Generalization in Energy-Based Models
• MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera
• ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal
• RIS-GAN: Explore Residual and Illumination with Generative Adversarial Networks for Shadow Removal
• InverseRenderNet: Learning single image inverse rendering
• Leveraging Multi-view Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation
• Semantic Hierarchical Priors for Intrinsic Image Decomposition
• Learning to Separate Multiple Illuminants in a Single Image
• CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering
• Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences