Depth-aware Blending of Smoothed Images for Bokeh Effect Generation
arXiv:2005.14214 · doi:10.1016/j.jvcir.2021.103089
Abstract
Bokeh effect is used in photography to capture images where the closer objects look sharp and every-thing else stays out-of-focus. Bokeh photos are generally captured using Single Lens Reflex cameras using shallow depth-of-field. Most of the modern smartphones can take bokeh images by leveraging dual rear cameras or a good auto-focus hardware. However, for smartphones with single-rear camera without a good auto-focus hardware, we have to rely on software to generate bokeh images. This kind of system is also useful to generate bokeh effect in already captured images. In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images. The original image and different versions of smoothed images are blended to generate Bokeh effect with the help of a monocular depth estimation network. The proposed approach is compared against a saliency detection based baseline and a number of approaches proposed in AIM 2019 Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order to understand different parts of the proposed algorithm. The network is lightweight and can process an HD image in 0.03 seconds. This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Conditional Random Fields as Recurrent Neural Networks
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Synthetic Depth-of-Field with a Single-Camera Mobile Phone
- Single-Image Depth Perception in the Wild
- Depth image denoising using nuclear norm and learning graph model