Physics-Based Generative Adversarial Models for Image Restoration and Beyond

Abstract

We present an algorithm to directly solve numerous image restoration problems (e.g., image deblurring, image dehazing, image deraining, etc.). These problems are highly ill-posed, and the common assumptions for existing methods are usually based on heuristic image priors. In this paper, we find that these problems can be solved by generative models with adversarial learning. However, the basic formulation of generative adversarial networks (GANs) does not generate realistic images, and some structures of the estimated images are usually not preserved well. Motivated by an interesting observation that the estimated results should be consistent with the observed inputs under the physics models, we propose a physics model constrained learning algorithm so that it can guide the estimation of the specific task in the conventional GAN framework. The proposed algorithm is trained in an end-to-end fashion and can be applied to a variety of image restoration and related low-level vision problems. Extensive experiments demonstrate that our method performs favorably against the state-of-the-art algorithms.

arXiv commentIEEE TPAMI

References in corpus (6)

• DehazeNet: An End-to-End System for Single Image Haze Removal
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Clearing the Skies: A deep network architecture for single-image rain removal
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Image De-raining Using a Conditional Generative Adversarial Network
• Amortised MAP Inference for Image Super-resolution

Cited by in corpus (11)

• Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles
• Blending Diverse Physical Priors with Neural Networks
• Automating Motion Correction in Multishot MRI Using Generative Adversarial Networks
• Deblurring by Realistic Blurring
• An Effective Two-Branch Model-Based Deep Network for Single Image Deraining
• Deep Shape from Polarization
• Blind Motion Deblurring with Cycle Generative Adversarial Networks
• Image Formation Model Guided Deep Image Super-Resolution
• Deep Likelihood Network for Image Restoration with Multiple Degradation Levels
• Modeling Deep Learning Based Privacy Attacks on Physical Mail
• Customized OCT images compression scheme with deep neural network