Marginal likelihood computation for model selection and hypothesis testing: an extensive review

Abstract

This is an up-to-date introduction to, and overview of, marginal likelihood computation for model selection and hypothesis testing. Computing normalizing constants of probability models (or ratio of constants) is a fundamental issue in many applications in statistics, applied mathematics, signal processing and machine learning. This article provides a comprehensive study of the state-of-the-art of the topic. We highlight limitations, benefits, connections and differences among the different techniques. Problems and possible solutions with the use of improper priors are also described. Some of the most relevant methodologies are compared through theoretical comparisons and numerical experiments.

Keywords: Marginal likelihood, Bayesian evidence, numerical integration, model selection, hypothesis testing, quadrature rules, double-intractable posteriors, partition functions

References in corpus (4)

• Cooperative Parallel Particle Filters for online model selection and applications to Urban Mobility
• Efficient Multiple Importance Sampling Estimators
• Compressed Monte Carlo with application in particle filtering
• Heretical Multiple Importance Sampling

Cited by in corpus (18)

• Revisiting Bayesian Autoencoders with MCMC
• Bayesian model comparison for simulation-based inference
• Spectral information criterion for automatic elbow detection
• BNNpriors: A library for Bayesian neural network inference with different prior distributions
• Adaptive quadrature schemes for Bayesian inference via active learning
• Universal and Automatic Elbow Detection for Learning the Effective Number of Components in Model Selection Problems
• MCMC-driven importance samplers
• Sparse Bayesian Estimation of Parameters in Linear-Gaussian State-Space Models
• An exhaustive variable selection study for linear models of soundscape emotions: rankings and Gibbs analysis
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
• Inference over radiative transfer models using variational and expectation maximization methods
• A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC
• An index of effective number of variables for uncertainty and reliability analysis in model selection problems
• Deep Importance Sampling based on Regression for Model Inversion and Emulation
• Adaptive posterior distributions for uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals
• A note on the area under the likelihood and the fake evidence for model selection
• Quantile Importance Sampling
• Evaluating marginal likelihood approximations of dose-response relationship models in Bayesian benchmark dose methods for risk assessment