Bridging Explicit and Implicit Deep Generative Models via Neural Stein Estimators

Abstract

There are two types of deep generative models: explicit and implicit. The former defines an explicit density form that allows likelihood inference; while the latter targets a flexible transformation from random noise to generated samples. While the two classes of generative models have shown great power in many applications, both of them, when used alone, suffer from respective limitations and drawbacks. To take full advantages of both models and enable mutual compensation, we propose a novel joint training framework that bridges an explicit (unnormalized) density estimator and an implicit sample generator via Stein discrepancy. We show that our method 1) induces novel mutual regularization via kernel Sobolev norm penalization and Moreau-Yosida regularization, and 2) stabilizes the training dynamics. Empirically, we demonstrate that proposed method can facilitate the density estimator to more accurately identify data modes and guide the generator to output higher-quality samples, comparing with training a single counterpart. The new approach also shows promising results when the training samples are contaminated or limited.

Accepted by NeurIPS2021 main conference

References in corpus (8)

• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• A Variational Inequality Perspective on Generative Adversarial Networks
• Implicit Generation and Generalization in Energy-Based Models
• Global Convergence to the Equilibrium of GANs using Variational Inequalities
• Calibrating Energy-based Generative Adversarial Networks
• Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE
• Convergence of Gradient Methods on Bilinear Zero-Sum Games
• Learning Implicit Generative Models by Teaching Explicit Ones