Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo

Abstract

Intractable generative models are models for which the likelihood is unavailable but sampling is possible. Most approaches to parameter inference in this setting require the computation of some discrepancy between the data and the generative model. This is for example the case for minimum distance estimation and approximate Bayesian computation. These approaches require sampling a high number of realisations from the model for different parameter values, which can be a significant challenge when simulating is an expensive operation. In this paper, we propose to enhance this approach by enforcing "sample diversity" in simulations of our models. This will be implemented through the use of quasi-Monte Carlo (QMC) point sets. Our key results are sample complexity bounds which demonstrate that, under smoothness conditions on the generator, QMC can significantly reduce the number of samples required to obtain a given level of accuracy when using three of the most common discrepancies: the maximum mean discrepancy, the Wasserstein distance, and the Sinkhorn divergence. This is complemented by a simulation study which highlights that an improved accuracy is sometimes also possible in some settings which are not covered by the theory.

arXiv commentAssumptions were weakened and a further numerical experiment was added

References in corpus (19)

• The frontier of simulation-based inference
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Generative Moment Matching Networks
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Training generative neural networks via Maximum Mean Discrepancy optimization
• Learning in Implicit Generative Models
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
• Approximate Bayesian computation with the Wasserstein distance
• Sample Complexity of Sinkhorn divergences
• Interpolating between Optimal Transport and MMD using Sinkhorn Divergences
• Approximate Bayesian Computation for Astronomical Model Analysis: A Case Study in Galaxy Demographics and Morphological Transformation at High Redshift
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• DR-ABC: Approximate Bayesian Computation with Kernel-Based Distribution Regression
• Kernel Recursive ABC: Point Estimation with Intractable Likelihood
• Generalized Posteriors in Approximate Bayesian Computation
• Generalized Sliced Distances for Probability Distributions
• Improving Approximate Optimal Transport Distances using Quantization
• A Scrambled Method of Moments