Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis

Abstract

We investigate the utility to computational Bayesian analyses of a particular family of recursive marginal likelihood estimators characterized by the (equivalent) algorithms known as "biased sampling" or "reverse logistic regression" in the statistics literature and "the density of states" in physics. Through a pair of numerical examples (including mixture modeling of the well-known galaxy data set) we highlight the remarkable diversity of sampling schemes amenable to such recursive normalization, as well as the notable efficiency of the resulting pseudo-mixture distributions for gauging prior sensitivity in the Bayesian model selection context. Our key theoretical contributions are to introduce a novel heuristic ("thermodynamic integration via importance sampling") for qualifying the role of the bridging sequence in this procedure and to reveal various connections between these recursive estimators and the nested sampling technique.

Published in at http://dx.doi.org/10.1214/13-STS465 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (9)

• Importance Nested Sampling and the MultiNest Algorithm
• Exact sampling and counting for fixed-margin matrices
• Gaussian Process Modelling of Asteroseismic Data
• Bayesian Inference on Mixtures of Distributions
• A Bayesian method for the analysis of deterministic and stochastic time series
• Consistency of the Adaptive Multiple Importance Sampling
• On the Evidence for Cosmic Variation of the Fine Structure Constant (I): A Parametric Bayesian Model Selection Analysis of the Quasar Dataset
• Symmetries of abelian ideals of Borel subalgebras
• On the Evidence for Cosmic Variation of the Fine Structure Constant (II): A Semi-Parametric Bayesian Model Selection Analysis of the Quasar Dataset

Cited by in corpus (31)

• Importance Nested Sampling and the MultiNest Algorithm
• Juliet: a versatile modelling tool for transiting and non-transiting exoplanetary systems
• Dynamic temperature selection for parallel-tempering in Markov chain Monte Carlo simulations
• Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation
• A statistical test for Nested Sampling algorithms
• TempoNest: A Bayesian approach to pulsar timing analysis
• Nested sampling for physical scientists
• Nested Sampling Methods
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review
• Quantifying the Bayesian Evidence for a Planet in Radial Velocity Data
• Overfitting Bayesian Mixture Models with an Unknown Number of Components
• Fast Bayesian Inference for Exoplanet Discovery in Radial Velocity Data
• Importance nested sampling with normalising flows
• Robust sampling for weak lensing and clustering analyses with the Dark Energy Survey
• Do joint CMB and HST data support a scale invariant spectrum?
• Kinematic Modelling of Disc Galaxies using Graphics Processing Units
• Testing non-minimally coupled inflation with CMB data: a Bayesian analysis
• CMB constraints on $\beta$-exponential inflationary models
• Bayesian Median Autoregression for Robust Time Series Forecasting
• On the safe use of prior densities for Bayesian model selection
• Bayesian analysis of inflationary features in Planck and SDSS data
• The ESO SupJup Survey II: The 12C/13C ratios of three young brown dwarfs with CRIRES${}^{+}$
• Exploring Photometric Redshifts as an Optimization Problem: An Ensemble MCMC and Simulated Annealing-Driven Template-Fitting Approach
• Facial Keypoints Detection
• Nested sampling for frequentist computation: fast estimation of small $p$-values
• On the Evidence for Cosmic Variation of the Fine Structure Constant (I): A Parametric Bayesian Model Selection Analysis of the Quasar Dataset
• Pulsar timing irregularities and the Neutron Star interior in the era of SKA: An Indian Outlook
• A Generalized Savage-Dickey Ratio
• A note on the area under the likelihood and the fake evidence for model selection
• Discussion of: "A Bayesian information criterion for singular models"
• Referenced Thermodynamic Integration for Bayesian Model Selection: Application to COVID-19 Model Selection