Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring

Abstract

Non-uniform blind deblurring for general dynamic scenes is a challenging computer vision problem as blurs arise not only from multiple object motions but also from camera shake, scene depth variation. To remove these complicated motion blurs, conventional energy optimization based methods rely on simple assumptions such that blur kernel is partially uniform or locally linear. Moreover, recent machine learning based methods also depend on synthetic blur datasets generated under these assumptions. This makes conventional deblurring methods fail to remove blurs where blur kernel is difficult to approximate or parameterize (e.g. object motion boundaries). In this work, we propose a multi-scale convolutional neural network that restores sharp images in an end-to-end manner where blur is caused by various sources. Together, we present multi-scale loss function that mimics conventional coarse-to-fine approaches. Furthermore, we propose a new large-scale dataset that provides pairs of realistic blurry image and the corresponding ground truth sharp image that are obtained by a high-speed camera. With the proposed model trained on this dataset, we demonstrate empirically that our method achieves the state-of-the-art performance in dynamic scene deblurring not only qualitatively, but also quantitatively.

References in corpus (8)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• FlowNet: Learning Optical Flow with Convolutional Networks
• Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal
• A Neural Approach to Blind Motion Deblurring
• Generalized Video Deblurring for Dynamic Scenes
• Dynamic Scene Deblurring using a Locally Adaptive Linear Blur Model

Cited by in corpus (19)

• DRFN: Deep Recurrent Fusion Network for Single-Image Super-Resolution with Large Factors
• DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
• Gated Fusion Network for Single Image Dehazing
• SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution
• Online Video Deblurring via Dynamic Temporal Blending Network
• Learning Digital Camera Pipeline for Extreme Low-Light Imaging
• Learning to Extract a Video Sequence from a Single Motion-Blurred Image
• Deep Semantic Face Deblurring
• Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera
• TOM-Net: Learning Transparent Object Matting from a Single Image
• Reblur2Deblur: Deblurring Videos via Self-Supervised Learning
• Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera using Deep Residual Networks
• Simultaneous Stereo Video Deblurring and Scene Flow Estimation
• Blind Image Deconvolution using Pretrained Generative Priors
• Deep Network Interpolation for Continuous Imagery Effect Transition
• Robust Blind Deconvolution via Mirror Descent
• Deep Learning with Inaccurate Training Data for Image Restoration
• Depth Map Completion by Jointly Exploiting Blurry Color Images and Sparse Depth Maps
• Cross-Scale Residual Network for Multiple Tasks:Image Super-resolution, Denoising, and Deblocking