Marginal likelihood computation for model selection and hypothesis testing: an extensive review

Abstract

This is an up-to-date introduction to, and overview of, marginal likelihood computation for model selection and hypothesis testing. Computing normalizing constants of probability models (or ratio of constants) is a fundamental issue in many applications in statistics, applied mathematics, signal processing and machine learning. This article provides a comprehensive study of the state-of-the-art of the topic. We highlight limitations, benefits, connections and differences among the different techniques. Problems and possible solutions with the use of improper priors are also described. Some of the most relevant methodologies are compared through theoretical comparisons and numerical experiments.

arXiv commentKeywords: Marginal likelihood, Bayesian evidence, numerical integration, model selection, hypothesis testing, quadrature rules, double-intractable posteriors, partition functions

References in corpus (22)

• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Comparison of Bayesian predictive methods for model selection
• Effective Sample Size for Importance Sampling based on discrepancy measures
• Generalized Multiple Importance Sampling
• Cooperative Parallel Particle Filters for online model selection and applications to Urban Mobility
• Properties of Nested Sampling
• Layered Adaptive Importance Sampling
• Improved Adaptive Rejection Metropolis Sampling Algorithms
• A Review of Multiple Try MCMC algorithms for Signal Processing
• Efficient Multiple Importance Sampling Estimators
• Computational methods for Bayesian model choice
• Group Importance Sampling for Particle Filtering and MCMC
• Minimum Description Length Revisited
• Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis
• On the flexibility of the design of Multiple Try Metropolis schemes
• Improving power posterior estimation of statistical evidence
• Compressed Monte Carlo with application in particle filtering
• Heretical Multiple Importance Sampling
• A multi-point Metropolis scheme with generic weight functions
• Metropolis Sampling
• Adaptive Independent Sticky MCMC algorithms
• On the correspondence between thermodynamics and inference

Cited by in corpus (18)

• Revisiting Bayesian Autoencoders with MCMC
• Bayesian model comparison for simulation-based inference
• Spectral information criterion for automatic elbow detection
• BNNpriors: A library for Bayesian neural network inference with different prior distributions
• Adaptive quadrature schemes for Bayesian inference via active learning
• Universal and Automatic Elbow Detection for Learning the Effective Number of Components in Model Selection Problems
• MCMC-driven importance samplers
• Sparse Bayesian Estimation of Parameters in Linear-Gaussian State-Space Models
• An exhaustive variable selection study for linear models of soundscape emotions: rankings and Gibbs analysis
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
• Inference over radiative transfer models using variational and expectation maximization methods
• A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC
• An index of effective number of variables for uncertainty and reliability analysis in model selection problems
• Deep Importance Sampling based on Regression for Model Inversion and Emulation
• Adaptive posterior distributions for uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals
• A note on the area under the likelihood and the fake evidence for model selection
• Quantile Importance Sampling
• Evaluating marginal likelihood approximations of dose-response relationship models in Bayesian benchmark dose methods for risk assessment