Decoupled Learning for Conditional Adversarial Networks

Abstract

Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks. We observe that the state-of-the-art achievements were obtained by carefully balancing the reconstruction loss and adversarial loss, and such balance shifts with different network structures, datasets, and training strategies. Empirical studies have demonstrated that an inappropriate weight between the two losses may cause instability, and it is tricky to search for the optimal setting, especially when lacking prior knowledge on the data and network. This paper gives the first attempt to relax the need of manual balancing by proposing the concept of \textit{decoupled learning}, where a novel network structure is designed that explicitly disentangles the backpropagation paths of the two losses. Experimental results demonstrate the effectiveness, robustness, and generality of the proposed method. The other contribution of the paper is the design of a new evaluation metric to measure the image quality of generative models. We propose the so-called \textit{normalized relative discriminative score} (NRDS), which introduces the idea of relative comparison, rather than providing absolute estimates like existing metrics.

References in corpus (10)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Improved Techniques for Training GANs
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Autoencoding beyond pixels using a learned similarity metric
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Least Squares Generative Adversarial Networks
• Age Progression/Regression by Conditional Adversarial Autoencoder
• r-BTN: Cross-domain Face Composite and Synthesis from Limited Facial Patches

Cited by in corpus (4)

• Talking Face Generation by Conditional Recurrent Adversarial Network
• Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations
• Quantifying point cloud realism through adversarially learned latent representations
• Towards quantitative methods to assess network generative models