Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning
arXiv:1902.02812
Abstract
This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initializer and slow thinking solver. The initializer generates the output directly by a non-linear transformation of the input as well as a noise vector that accounts for latent variability in the output. The slow thinking solver learns an objective function in the form of a conditional energy function, so that the output can be generated by optimizing the objective function, or more rigorously by sampling from the conditional energy-based model. We propose to learn the two models jointly, where the fast thinking initializer serves to initialize the sampling of the slow thinking solver, and the solver refines the initial output by an iterative algorithm. The solver learns from the difference between the refined output and the observed output, while the initializer learns from how the solver refines its initial output. We demonstrate the effectiveness of the proposed method on various conditional learning tasks, e.g., class-to-image generation, image-to-image translation, and image recovery. The advantage of our method over GAN-based methods is that our method is equipped with a slow thinking process that refines the solution guided by a learned objective function.
arXiv comment16 pages
References in corpus (20)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Auto-Encoding Variational Bayes
- Conditional Generative Adversarial Nets
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Pixel Recurrent Neural Networks
- NICE: Non-linear Independent Components Estimation
- Toward Multimodal Image-to-Image Translation
- Energy-based Generative Adversarial Network
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Deep Generative Stochastic Networks Trainable by Backprop
- Better Mixing via Deep Representations
- A Theory of Generative ConvNet
- Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model
- Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
- Alternating Back-Propagation for Generator Network
- Generative Latent Flow
- Variational Conditional GAN for Fine-grained Controllable Image Generation