A Dark Flash Normal Camera
arXiv:2012.06125
Abstract
Casual photography is often performed in uncontrolled lighting that can result in low quality images and degrade the performance of downstream processing. We consider the problem of estimating surface normal and reflectance maps of scenes depicting people despite these conditions by supplementing the available visible illumination with a single near infrared (NIR) light source and camera, a so-called "dark flash image". Our method takes as input a single color image captured under arbitrary visible lighting and a single dark flash image captured under controlled front-lit NIR lighting at the same viewpoint, and computes a normal map, a diffuse albedo map, and a specular intensity map of the scene. Since ground truth normal and reflectance maps of faces are difficult to capture, we propose a novel training technique that combines information from two readily available and complementary sources: a stereo depth signal and photometric shading cues. We evaluate our method over a range of subjects and lighting conditions and describe two applications: optimizing stereo geometry and filling the shadows in an image.
References in corpus (7)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Single Image Portrait Relighting
- Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
- Monocular Depth Estimation: A Survey
- Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision
- Towards Geometry Guided Neural Relighting with Flash Photography