Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy

Abstract

We propose a method to optimize the representation and distinguishability of samples from two probability distributions, by maximizing the estimated power of a statistical test based on the maximum mean discrepancy (MMD). This optimized MMD is applied to the setting of unsupervised learning by generative adversarial networks (GAN), in which a model attempts to generate realistic samples, and a discriminator attempts to tell these apart from data samples. In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples. Second, the MMD can be used to evaluate the performance of a generative model, by testing the model's samples against a reference data set. In the latter role, the optimized MMD is particularly helpful, as it gives an interpretable indication of how the model and data distributions differ, even in cases where individual model samples are not easily distinguished either by eye or by classifier.

Published at ICLR 2017 (public comments: http://openreview.net/forum?id=HJWHIKqgl )

Cited by in corpus (64)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Demystifying MMD GANs
• PI-VAE: Physics-Informed Variational Auto-Encoder for stochastic differential equations
• Approximation and Convergence Properties of Generative Adversarial Learning
• On gradient regularizers for MMD GANs
• Head2Head++: Deep Facial Attributes Re-Targeting
• Generative adversarial networks in time series: A survey and taxonomy
• A Linear-Time Kernel Goodness-of-Fit Test
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Discovering and Deciphering Relationships Across Disparate Data Modalities
• A Kernel Two-Sample Test for Functional Data
• Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling
• Airline Passenger Name Record Generation using Generative Adversarial Networks
• Synthesis of Realistic ECG using Generative Adversarial Networks
• Distributionally Robust Bayesian Optimization
• Adversarial Time-to-Event Modeling
• On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternatives
• How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models
• Metrizing Weak Convergence with Maximum Mean Discrepancies
• Distributionally Robust Optimization and Generalization in Kernel Methods
• Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning
• Kernel Change-point Detection with Auxiliary Deep Generative Models
• Sliced Kernelized Stein Discrepancy
• Quantitatively Evaluating GANs With Divergences Proposed for Training
• Methods for Quantifying Dataset Similarity: a Review, Taxonomy and Comparison
• MMD Aggregated Two-Sample Test
• TripletGAN: Training Generative Model with Triplet Loss
• Two Sample Testing in High Dimension via Maximum Mean Discrepancy
• Learning Kernel Tests Without Data Splitting
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• Classification Logit Two-sample Testing by Neural Networks
• Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift
• Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes
• Learning Implicit Generative Models with the Method of Learned Moments
• Human Motion Analysis with Deep Metric Learning
• Two-sample Test with Kernel Projected Wasserstein Distance
• Kernel Stein Generative Modeling
• Deep Dimension Reduction for Supervised Representation Learning
• Learning Implicit Generative Models with Theoretical Guarantees
• Deep Generative Learning via Variational Gradient Flow
• Unbiased estimators for the variance of MMD estimators
• Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions
• Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit
• Statistical Depth Meets Machine Learning: Kernel Mean Embeddings and Depth in Functional Data Analysis
• Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo
• Testing Goodness of Fit of Conditional Density Models with Kernels
• Kernel-Guided Training of Implicit Generative Models with Stability Guarantees
• Generative Learning With Euler Particle Transport
• Revealing posturographic features associated with the risk of falling in patients with Parkinsonian syndromes via machine learning
• Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
• Asymptotically Optimal One- and Two-Sample Testing with Kernels
• Kernel Hypothesis Testing with Set-valued Data
• Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator
• Exponentially Consistent Kernel Two-Sample Tests
• Easy High-Dimensional Likelihood-Free Inference
• Kernel Mean Matching for Content Addressability of GANs
• Irregular Convolutional Auto-Encoder on Point Clouds
• On the Optimization Landscape of Maximum Mean Discrepancy
• A Witness Two-Sample Test
• A fast and effective kernel two-sample test for large-scale data
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement