Non-Local Video Denoising by CNN
arXiv:1811.12758
Abstract
Non-local patch based methods were until recently state-of-the-art for image denoising but are now outperformed by CNNs. Yet they are still the state-of-the-art for video denoising, as video redundancy is a key factor to attain high denoising performance. The problem is that CNN architectures are hardly compatible with the search for self-similarities. In this work we propose a new and efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed architecture to image and video denoising. For the latter patches are searched for in a 3D spatio-temporal volume. The proposed architecture achieves state-of-the-art results. To the best of our knowledge, this is the first successful application of a CNN to video denoising.
A shorter version of this work has been accepted at ICIP 2019 (A NON-LOCAL CNN FOR VIDEO DENOISING). The results of v2 were improved compared to v1 and the code was updated accordingly. Code is available at: https://github.com/axeldavy/vnlnet
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
- Nonlocality-Reinforced Convolutional Neural Networks for Image Denoising
- Generalized Deep Image to Image Regression
Cited by in corpus (5)
- Deep Learning on Image Denoising: An overview
- Real-time Streaming Video Denoising with Bidirectional Buffers
- First image then video: A two-stage network for spatiotemporal video denoising
- A Lightweight Structure Aimed to Utilize Spatial Correlation for Sparse-View CT Reconstruction
- Implementation of the VBM3D Video Denoising Method and Some Variants