Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning

Abstract

We investigate a novel approach for image restoration by reinforcement learning. Unlike existing studies that mostly train a single large network for a specialized task, we prepare a toolbox consisting of small-scale convolutional networks of different complexities and specialized in different tasks. Our method, RL-Restore, then learns a policy to select appropriate tools from the toolbox to progressively restore the quality of a corrupted image. We formulate a step-wise reward function proportional to how well the image is restored at each step to learn the action policy. We also devise a joint learning scheme to train the agent and tools for better performance in handling uncertainty. In comparison to conventional human-designed networks, RL-Restore is capable of restoring images corrupted with complex and unknown distortions in a more parameter-efficient manner using the dynamically formed toolchain.

arXiv commentTo appear at CVPR 2018 (Spotlight). Project page: http://mmlab.ie.cuhk.edu.hk/projects/RL-Restore/

References in corpus (17)

• Distilling the Knowledge in a Neural Network
• Continuous control with deep reinforcement learning
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Neural Architecture Search with Reinforcement Learning
• Recurrent Models of Visual Attention
• Compressing Neural Networks with the Hashing Trick
• MemNet: A Persistent Memory Network for Image Restoration
• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal
• Deep Reinforcement Learning-based Image Captioning with Embedding Reward
• Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
• Learning Policies for Adaptive Tracking with Deep Feature Cascades
• On learning optimized reaction diffusion processes for effective image restoration
• Attention-Aware Face Hallucination via Deep Reinforcement Learning
• One-to-Many Network for Visually Pleasing Compression Artifacts Reduction
• Merge or Not? Learning to Group Faces via Imitation Learning
• Learning Dynamic Hierarchical Models for Anytime Scene Labeling

Cited by in corpus (5)

• EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
• ALBA : Reinforcement Learning for Video Object Segmentation
• Learning image from projection: a full-automatic reconstruction (FAR) net for sparse-views computed tomography
• Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning
• Sparsely ensembled convolutional neural network classifiers via reinforcement learning