Path-Restore: Learning Network Path Selection for Image Restoration

Abstract

Very deep Convolutional Neural Networks (CNNs) have greatly improved the performance on various image restoration tasks. However, this comes at a price of increasing computational burden, hence limiting their practical usages. We observe that some corrupted image regions are inherently easier to restore than others since the distortion and content vary within an image. To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region. We train the pathfinder using reinforcement learning with a difficulty-regulated reward. This reward is related to the performance, complexity and "the difficulty of restoring a region". A policy mask is further investigated to jointly process all the image regions. We conduct experiments on denoising and mixed restoration tasks. The results show that our method achieves comparable or superior performance to existing approaches with less computational cost. In particular, Path-Restore is effective for real-world denoising, where the noise distribution varies across different regions on a single image. Compared to the state-of-the-art RIDNet, our method achieves comparable performance and runs 2.7x faster on the realistic Darmstadt Noise Dataset.

IEEE TPAMI 2021. Project page: https://www.mmlab-ntu.com/project/pathrestore/

References in corpus (6)

• Neural Architecture Search with Reinforcement Learning
• FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• Designing Neural Network Architectures using Reinforcement Learning
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Exposure: A White-Box Photo Post-Processing Framework

Cited by in corpus (17)

• Dynamic Path-Controllable Deep Unfolding Network for Compressive Sensing
• Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data
• EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
• Towards Real-World Blind Face Restoration with Generative Facial Prior
• When AWGN-based Denoiser Meets Real Noises
• Interactive Multi-Dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration
• Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems
• Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy
• Progressive Training of Multi-level Wavelet Residual Networks for Image Denoising
• Multi-frame Joint Enhancement for Early Interlaced Videos
• TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems
• H-OWAN: Multi-distorted Image Restoration with Tensor 1x1 Convolution
• R3L: Connecting Deep Reinforcement Learning to Recurrent Neural Networks for Image Denoising via Residual Recovery
• Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
• Deep Adaptive Inference Networks for Single Image Super-Resolution
• Pyramid Real Image Denoising Network
• Sparsely ensembled convolutional neural network classifiers via reinforcement learning