Bayesian Cycle-Consistent Generative Adversarial Networks via Marginalizing Latent Sampling

Abstract

Recent techniques built on Generative Adversarial Networks (GANs), such as Cycle-Consistent GANs, are able to learn mappings among different domains built from unpaired datasets, through min-max optimization games between generators and discriminators. However, it remains challenging to stabilize the training process and thus cyclic models fall into mode collapse accompanied by the success of discriminator. To address this problem, we propose an novel Bayesian cyclic model and an integrated cyclic framework for inter-domain mappings. The proposed method motivated by Bayesian GAN explores the full posteriors of cyclic model via sampling latent variables and optimizes the model with maximum a posteriori (MAP) estimation. Hence, we name it Bayesian CycleGAN. In addition, original CycleGAN cannot generate diversified results. But it is feasible for Bayesian framework to diversify generated images by replacing restricted latent variables in inference process. We evaluate the proposed Bayesian CycleGAN on multiple benchmark datasets, including Cityscapes, Maps, and Monet2photo. The proposed method improve the per-pixel accuracy by 15% for the Cityscapes semantic segmentation task within origin framework and improve 20% within the proposed integrated framework, showing better resilience to imbalance confrontation. The diversified results of Monet2Photo style transfer also demonstrate its superiority over original cyclic model. We provide codes for all of our experiments in https://github.com/ranery/Bayesian-CycleGAN.

arXiv commentAccepted by IEEE TNNLS

References in corpus (12)

• Auto-Encoding Variational Bayes
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation
• Don't Decay the Learning Rate, Increase the Batch Size
• Stochastic Gradient Hamiltonian Monte Carlo
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond
• Bayesian GAN
• Bayesian Generative Active Deep Learning
• Generative Adversarial Networks as Variational Training of Energy Based Models