Generalized Bayesian Likelihood-Free Inference
arXiv:2104.03889 · doi:10.1214/24-EJS2283
Abstract
We propose a posterior for Bayesian Likelihood-Free Inference (LFI) based on generalized Bayesian inference. To define the posterior, we use Scoring Rules (SRs), which evaluate probabilistic models given an observation. In LFI, we can sample from the model but not evaluate the likelihood; hence, we employ SRs which admit unbiased empirical estimates. We use the Energy and Kernel SRs, for which our posterior enjoys consistency in a well-specified setting and outlier robustness. We perform inference with pseudo-marginal (PM) Markov Chain Monte Carlo (MCMC) or stochastic-gradient (SG) MCMC. While PM-MCMC works satisfactorily for simple setups, it mixes poorly for concentrated targets. Conversely, SG-MCMC requires differentiating the simulator model but improves performance over PM-MCMC when both work and scales to higher-dimensional setups as it is rejection-free. Although both techniques target the SR posterior approximately, the error diminishes as the number of model simulations at each MCMC step increases. In our simulations, we employ automatic differentiation to effortlessly differentiate the simulator model. We compare our posterior with related approaches on standard benchmarks and a chaotic dynamical system from meteorology, for which SG-MCMC allows inferring the parameters of a neural network used to parametrize a part of the update equations of the dynamical system.
arXiv comment21 pages, plus references and appendices. 21 figures. Code available at https://github.com/Shermjj/GenBayes_LikelihoodFree_ScoringRules_SGMCMC
References in corpus (30)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- The pseudo-marginal approach for efficient Monte Carlo computations
- A General Framework for Updating Belief Distributions
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Approximate Bayesian computation with the Wasserstein distance
- Theory and Applications of Proper Scoring Rules
- Bayesian Indirect Inference Using a Parametric Auxiliary Model
- Calibrating general posterior credible regions
- Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions
- Principles of Bayesian Inference using General Divergence Criteria
- Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo
- Statistical Inference for Generative Models with Maximum Mean Discrepancy
- MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
- Adaptive Thermostats for Noisy Gradient Systems
- K2-ABC: Approximate Bayesian Computation with Kernel Embeddings
- Objective Bayesian inference with proper scoring rules
- Asymptotic normality, concentration, and coverage of generalized posteriors
- Assigning a value to a power likelihood in a general Bayesian model
- Approximate Bayesian Computation with composite score functions
- Accelerating pseudo-marginal Metropolis-Hastings by correlating auxiliary variables
- Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap
- ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation
- Bayesian Learning of Parameterised Quantum Circuits
- Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computation
- Generalized Bayesian Likelihood-Free Inference
- BSL: An R Package for Efficient Parameter Estimation for Simulation-Based Models via Bayesian Synthetic Likelihood
- Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoods
- Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap
- Reliable Bayesian Inference in Misspecified Models