Self-Supervised Intrinsic Image Decomposition

Abstract

Intrinsic decomposition from a single image is a highly challenging task, due to its inherent ambiguity and the scarcity of training data. In contrast to traditional fully supervised learning approaches, in this paper we propose learning intrinsic image decomposition by explaining the input image. Our model, the Rendered Intrinsics Network (RIN), joins together an image decomposition pipeline, which predicts reflectance, shape, and lighting conditions given a single image, with a recombination function, a learned shading model used to recompose the original input based off of intrinsic image predictions. Our network can then use unsupervised reconstruction error as an additional signal to improve its intermediate representations. This allows large-scale unlabeled data to be useful during training, and also enables transferring learned knowledge to images of unseen object categories, lighting conditions, and shapes. Extensive experiments demonstrate that our method performs well on both intrinsic image decomposition and knowledge transfer.

NIPS 2017 camera-ready version, project page: http://rin.csail.mit.edu/

References in corpus (5)

• Deep Convolutional Inverse Graphics Network
• MarrNet: 3D Shape Reconstruction via 2.5D Sketches
• Deep Shading: Convolutional Neural Networks for Screen-Space Shading
• Deep Lambertian Networks
• Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression

Cited by in corpus (24)

• Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
• Intrinsic Image Decomposition via Ordinal Shading
• MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera
• Learning Intrinsic Image Decomposition from Watching the World
• Real-Time Global Illumination Decomposition of Videos
• Coherent, super resolved radar beamforming using self-supervised learning
• InverseRenderNet: Learning single image inverse rendering
• 3D Shape Perception from Monocular Vision, Touch, and Shape Priors
• Learning to Reconstruct Texture-less Deformable Surfaces from a Single View
• Unsupervised Learning for Intrinsic Image Decomposition from a Single Image
• Leveraging Multi-view Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation
• Robust Point Light Source Estimation Using Differentiable Rendering
• Face Inverse Rendering via Hierarchical Decoupling
• A General Albedo Recovery Approach for Aerial Photogrammetric Images through Inverse Rendering
• Intrinsic Image Transfer for Illumination Manipulation
• Deeply Learned Spectral Total Variation Decomposition
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
• Separate In Latent Space: Unsupervised Single Image Layer Separation
• An Optical physics inspired CNN approach for intrinsic image decomposition
• De-rendering the World's Revolutionary Artefacts
• Learning Intrinsic Images for Clothing
• Fast Fourier Intrinsic Network
• Unsupervised Deep Single-Image Intrinsic Decomposition using Illumination-Varying Image Sequences
• Intrinsic Image Decomposition using Paradigms