Multi-Stage Progressive Image Restoration

Abstract

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.

Accepted at CVPR 2021

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Transformers in Vision: A Survey
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• ParseNet: Looking Wider to See Better
• Clearing the Skies: A deep network architecture for single-image rain removal
• Variational Denoising Network: Toward Blind Noise Modeling and Removal
• Residual Non-local Attention Networks for Image Restoration
• Image Super-Resolution Using Deep Convolutional Networks
• DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
• Rethinking on Multi-Stage Networks for Human Pose Estimation

Cited by in corpus (7)

• Intriguing Findings of Frequency Selection for Image Deblurring
• HINet: Half Instance Normalization Network for Image Restoration
• SDNet: mutil-branch for single image deraining using swin
• SDWNet: A Straight Dilated Network with Wavelet Transformation for Image Deblurring
• Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report
• IFR: Iterative Fusion Based Recognizer For Low Quality Scene Text Recognition
• Blind Image Decomposition