Perceptual Adversarial Networks for Image-to-Image Transformation

Abstract

In this paper, we propose a principled Perceptual Adversarial Networks (PAN) for image-to-image transformation tasks. Unlike existing application-specific algorithms, PAN provides a generic framework of learning mapping relationship between paired images (Fig. 1), such as mapping a rainy image to its de-rained counterpart, object edges to its photo, semantic labels to a scenes image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the image transformation network T and the discriminative network D. Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve image-to-image transformation tasks. Among them, the hidden layers and output of the discriminative network D are upgraded to continually and automatically discover the discrepancy between the transformed image and the corresponding ground-truth. Simultaneously, the image transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through the adversarial training process, the image transformation network T will continually narrow the gap between transformed images and ground-truth images. Experiments evaluated on several image-to-image transformation tasks (e.g., image de-raining, image inpainting, etc.) show that the proposed PAN outperforms many related state-of-the-art methods.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Spectral Normalization for Generative Adversarial Networks
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Energy-based Generative Adversarial Network
• cGANs with Projection Discriminator
• Image De-raining Using a Conditional Generative Adversarial Network
• Neural Photo Editing with Introspective Adversarial Networks

Cited by in corpus (54)

• How Generative Adversarial Networks and Their Variants Work: An Overview
• Image inpainting: A review
• On Data Augmentation for GAN Training
• Cross-Modality Deep Feature Learning for Brain Tumor Segmentation
• Experimental Quantum Generative Adversarial Networks for Image Generation
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Image De-raining Using a Conditional Generative Adversarial Network
• Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation
• Unsupervised Medical Image Translation Using Cycle-MedGAN
• Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning
• Generating Visually Aligned Sound from Videos
• Adversarial Inpainting of Medical Image Modalities
• A Systematic Survey of Regularization and Normalization in GANs
• PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation
• Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
• PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier
• Image Fine-grained Inpainting
• ipA-MedGAN: Inpainting of Arbitrary Regions in Medical Imaging
• PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
• Efficient CNN-based Super Resolution Algorithms for mmWave Mobile Radar Imaging
• An Adversarial Super-Resolution Remedy for Radar Design Trade-offs
• Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space with Softplus Loss
• GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields
• AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection
• Constrained CycleGAN for Effective Generation of Ultrasound Sector Images of Improved Spatial Resolution
• DeViT: Deformed Vision Transformers in Video Inpainting
• Content and Colour Distillation for Learning Image Translations with the Spatial Profile Loss
• CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
• Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation
• GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernet
• Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?
• Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN
• Image Inpainting Guided by Coherence Priors of Semantics and Textures
• Improve Object Detection by Data Enhancement based on Generative Adversarial Nets
• Chinese Typeface Transformation with Hierarchical Adversarial Network
• DUAL-GLOW: Conditional Flow-Based Generative Model for Modality Transfer
• Semi-Supervised Video Deraining with Dynamical Rain Generator
• Deep Attentive Generative Adversarial Network for Photo-Realistic Image De-Quantization
• Training Generative Adversarial Networks with Adaptive Composite Gradient
• Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence
• Generative Smoke Removal
• High-dimensional Assisted Generative Model for Color Image Restoration
• Generative and Discriminative Learning for Distorted Image Restoration
• Towards Real-Time Advancement of Underwater Visual Quality with GAN
• Specular-to-Diffuse Translation for Multi-View Reconstruction
• A Novel Measure to Evaluate Generative Adversarial Networks Based on Direct Analysis of Generated Images
• HRVGAN: High Resolution Video Generation using Spatio-Temporal GAN
• MixCon: Adjusting the Separability of Data Representations for Harder Data Recovery
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection
• Association: Remind Your GAN not to Forget
• Analysis of Macula on Color Fundus Images Using Heightmap Reconstruction Through Deep Learning
• Quality-aware Unpaired Image-to-Image Translation
• Contrastive Feature Loss for Image Prediction
• Neural Wireframe Renderer: Learning Wireframe to Image Translations