Generalized Bayesian Likelihood-Free Inference

Abstract

We propose a posterior for Bayesian Likelihood-Free Inference (LFI) based on generalized Bayesian inference. To define the posterior, we use Scoring Rules (SRs), which evaluate probabilistic models given an observation. In LFI, we can sample from the model but not evaluate the likelihood; hence, we employ SRs which admit unbiased empirical estimates. We use the Energy and Kernel SRs, for which our posterior enjoys consistency in a well-specified setting and outlier robustness. We perform inference with pseudo-marginal (PM) Markov Chain Monte Carlo (MCMC) or stochastic-gradient (SG) MCMC. While PM-MCMC works satisfactorily for simple setups, it mixes poorly for concentrated targets. Conversely, SG-MCMC requires differentiating the simulator model but improves performance over PM-MCMC when both work and scales to higher-dimensional setups as it is rejection-free. Although both techniques target the SR posterior approximately, the error diminishes as the number of model simulations at each MCMC step increases. In our simulations, we employ automatic differentiation to effortlessly differentiate the simulator model. We compare our posterior with related approaches on standard benchmarks and a chaotic dynamical system from meteorology, for which SG-MCMC allows inferring the parameters of a neural network used to parametrize a part of the update equations of the dynamical system.

21 pages, plus references and appendices. 21 figures. Code available at https://github.com/Shermjj/GenBayes_LikelihoodFree_ScoringRules_SGMCMC

References in corpus (14)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• The pseudo-marginal approach for efficient Monte Carlo computations
• Approximate Bayesian computation with the Wasserstein distance
• Bayesian Indirect Inference Using a Parametric Auxiliary Model
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
• Assigning a value to a power likelihood in a general Bayesian model
• Bayesian Learning of Parameterised Quantum Circuits
• Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation
• Generalized Bayesian Likelihood-Free Inference
• BSL: An R Package for Efficient Parameter Estimation for Simulation-Based Models via Bayesian Synthetic Likelihood
• Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoods
• Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap
• Reliable Bayesian Inference in Misspecified Models

Cited by in corpus (1)

• Generalized Bayesian Likelihood-Free Inference