HDR image reconstruction from a single exposure using deep CNNs
arXiv:1710.07480 · doi:10.1145/3130800.3130816
Abstract
Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that have been lost in saturated image areas, in order to enable HDR reconstruction from a single exposure. We show that this problem is well-suited for deep learning algorithms, and propose a deep convolutional neural network (CNN) that is specifically designed taking into account the challenges in predicting HDR values. To train the CNN we gather a large dataset of HDR images, which we augment by simulating sensor saturation for a range of cameras. To further boost robustness, we pre-train the CNN on a simulated HDR dataset created from a subset of the MIT Places database. We demonstrate that our approach can reconstruct high-resolution visually convincing HDR results in a wide range of situations, and that it generalizes well to reconstruction of images captured with arbitrary and low-end cameras that use unknown camera response functions and post-processing. Furthermore, we compare to existing methods for HDR expansion, and show high quality results also for image based lighting. Finally, we evaluate the results in a subjective experiment performed on an HDR display. This shows that the reconstructed HDR images are visually convincing, with large improvements as compared to existing methods.
15 pages, 19 figures, Siggraph Asia 2017. Project webpage located at http://hdrv.org/hdrcnn/ where paper with high quality images is available, as well as supplementary material (document, images, video and source code)
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Fully Convolutional Networks for Semantic Segmentation
- Compression Artifacts Removal Using Convolutional Neural Networks
- Convolutional Sparse Coding for High Dynamic Range Imaging
- High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
- Evaluating the Performance of Existing Full-Reference Quality Metrics on High Dynamic Range (HDR) Video Content
- Learning High Dynamic Range from Outdoor Panoramas
Cited by in corpus (80)
- HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions
- Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss
- Deep Tone Mapping Operator for High Dynamic Range Images
- Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image
- Illumination-Adaptive Person Re-identification
- Spatial-Angular Attention Network for Light Field Reconstruction
- Learning to Adapt to Light
- Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image Restoration
- A leak in PRNU based source identification. Questioning fingerprint uniqueness
- HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal Reconstruction
- Single-frame Regularization for Temporally Stable CNNs
- A Little Bit More: Bitplane-Wise Bit-Depth Recovery
- DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering
- CycleISP: Real Image Restoration via Improved Data Synthesis
- CameraNet: A Two-Stage Framework for Effective Camera ISP Learning
- DeepFlash: Turning a Flash Selfie into a Studio Portrait
- NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
- Deep Graph Learning for Spatially-Varying Indoor Lighting Prediction
- Hybrid Loss for Learning Single-Image-based HDR Reconstruction
- Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models
- Deep HDR Hallucination for Inverse Tone Mapping
- Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting
- Attention-guided Network for Ghost-free High Dynamic Range Imaging
- Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline
- Road Context-aware Intrusion Detection System for Autonomous Cars
- Fearless Luminance Adaptation: A Macro-Micro-Hierarchical Transformer for Exposure Correction
- LHDR: HDR Reconstruction for Legacy Content using a Lightweight DNN
- Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping
- Modeling and Enhancing Low-quality Retinal Fundus Images
- Dual Illumination Estimation for Robust Exposure Correction
- Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications
- Image Correction via Deep Reciprocating HDR Transformation
- ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging
- Color-wise Attention Network for Low-light Image Enhancement
- HDRT: A Large-Scale Dataset for Infrared-Guided HDR Imaging
- HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization
- Robust estimation of exposure ratios in multi-exposure image stacks
- Switchable Temporal Propagation Network
- Synthetic Defocus and Look-Ahead Autofocus for Casual Videography
- JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video
- HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models
- Deep Inverse Tone Mapping Using LDR Based Learning for Estimating HDR Images with Absolute Luminance
- Diversifying Semantic Image Synthesis and Editing via Class- and Layer-wise VAEs
- Deep Reformulated Laplacian Tone Mapping
- MetaHDR: Model-Agnostic Meta-Learning for HDR Image Reconstruction
- HDRVideo-GAN: Deep Generative HDR Video Reconstruction
- Geological modeling using a recursive convolutional neural networks approach
- Deep Attentive Generative Adversarial Network for Photo-Realistic Image De-Quantization
- Redistributing the Precision and Content in 3D-LUT-based Inverse Tone-mapping for HDR/WCG Display
- Semantic Aware Diffusion Inverse Tone Mapping
- Image Demoireing with Learnable Bandpass Filters
- High Dynamic Range Image Reconstruction via Deep Explicit Polynomial Curve Estimation
- HDR-cGAN: Single LDR to HDR Image Translation using Conditional GAN
- Deep Learning for HDR Imaging: State-of-the-Art and Future Trends
- Lightness Modulated Deep Inverse Tone Mapping
- Neural Imaging Pipelines - the Scourge or Hope of Forensics?
- Image Enhancement Network Trained by Using HDR images
- Learning a Reinforced Agent for Flexible Exposure Bracketing Selection
- Warwick Image Forensics Dataset for Device Fingerprinting In Multimedia Forensics
- A Two-stage Deep Network for High Dynamic Range Image Reconstruction
- Invertible Image Signal Processing
- Beyond Visual Attractiveness: Physically Plausible Single Image HDR Reconstruction for Spherical Panoramas
- Luminance Attentive Networks for HDR Image and Panorama Reconstruction
- Progressive and Selective Fusion Network for High Dynamic Range Imaging
- High Dynamic Range Imaging via Visual Attention Modules
- Deep Snapshot HDR Reconstruction Based on the Polarization Camera
- Spectrally Consistent UNet for High Fidelity Image Transformations
- Optimal HDR and Depth from Dual Cameras
- A Dual Camera System for High Spatiotemporal Resolution Video Acquisition
- Estimation of Camera Response Function using Prediction Consistency and Gradual Refinement with an Extension to Deep Learning
- Enhancing HDR Video Compression through CNN-based Effective Bit Depth Adaptation
- Learning to Harmonize Cross-vendor X-ray Images by Non-linear Image Dynamics Correction
- Learning Reference-Guided Exposure Correction with Hybrid Illumination Characteristics
- Stay Positive: Non-Negative Image Synthesis for Augmented Reality
- Neural Illumination: Lighting Prediction for Indoor Environments
- Variable Augmented Network for Invertible Modality Synthesis-Fusion
- Invertible Tone Mapping with Selectable Styles
- Deep Snapshot HDR Imaging Using Multi-Exposure Color Filter Array
- A Shift-insensitive Full Reference Image Quality Assessment Model Based on Quadratic Sum of Gradient Magnitude and LOG signals
- Deep Controllable Backlight Dimming