Variational f-divergence Minimization
arXiv:1907.11891
Abstract
Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks, alternative non-likelihood training criteria have been proposed. Whilst not necessarily statistically efficient, these alternatives may better match user requirements such as sharp image generation. A general variational method for training probabilistic latent variable models using maximum likelihood is well established; however, how to train latent variable models using other f-divergences is comparatively unknown. We discuss a variational approach that, when combined with the recently introduced Spread Divergence, can be applied to train a large class of latent variable models using any f-divergence.
References in corpus (12)
- Rényi Divergence and Kullback-Leibler Divergence
- NIPS 2016 Tutorial: Generative Adversarial Networks
- Adversarial Feature Learning
- Estimating divergence functionals and the likelihood ratio by convex risk minimization
- Adversarially Learned Inference
- Towards Principled Methods for Training Generative Adversarial Networks
- Auxiliary Deep Generative Models
- Learning in Implicit Generative Models
- Variational Inference using Implicit Distributions
- Rényi Divergence Variational Inference
- Improved generator objectives for GANs
- Augment and Reduce: Stochastic Inference for Large Categorical Distributions