Kernel-Guided Training of Implicit Generative Models with Stability Guarantees
arXiv:1901.09206
Abstract
Modern implicit generative models such as generative adversarial networks (GANs) are generally known to suffer from issues such as instability, uninterpretability, and difficulty in assessing their performance. If we see these implicit models as dynamical systems, some of these issues are caused by being unable to control their behavior in a meaningful way during the course of training. In this work, we propose a theoretically grounded method to guide the training trajectories of GANs by augmenting the GAN loss function with a kernel-based regularization term that controls local and global discrepancies between the model and true distributions. This control signal allows us to inject prior knowledge into the model. We provide theoretical guarantees on the stability of the resulting dynamical system and demonstrate different aspects of it via a wide range of experiments.
This article supersedes arXiv:1901.09206 version 1. The paper is restructured, its writing is improved, and new experiments are added. The main result on stability is unchanged
References in corpus (10)
- Conditional Generative Adversarial Nets
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Energy-based Generative Adversarial Network
- Generative Moment Matching Networks
- Training generative neural networks via Maximum Mean Discrepancy optimization
- Fast Two-Sample Testing with Analytic Representations of Probability Measures
- On gradient regularizers for MMD GANs
- Improving MMD-GAN Training with Repulsive Loss Function
- Deep Generative Models with Learnable Knowledge Constraints
- Tempered Adversarial Networks