A General Framework for Updating Belief Distributions

Abstract

We propose a framework for general Bayesian inference. We argue that a valid update of a prior belief distribution to a posterior can be made for parameters which are connected to observations through a loss function rather than the traditional likelihood function, which is recovered under the special case of using self information loss. Modern application areas make it is increasingly challenging for Bayesians to attempt to model the true data generating mechanism. Moreover, when the object of interest is low dimensional, such as a mean or median, it is cumbersome to have to achieve this via a complete model for the whole data distribution. More importantly, there are settings where the parameter of interest does not directly index a family of density functions and thus the Bayesian approach to learning about such parameters is currently regarded as problematic. Our proposed framework uses loss-functions to connect information in the data to functionals of interest. The updating of beliefs then follows from a decision theoretic approach involving cumulative loss functions. Importantly, the procedure coincides with Bayesian updating when a true likelihood is known, yet provides coherent subjective inference in much more general settings. Connections to other inference frameworks are highlighted.

This is the pre-peer reviewed version of the article "A General Framework for Updating Belief Distributions", which has been accepted for publication in the Journal of Statistical Society - Series B. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving

References in corpus (4)

• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers
• On Bayesian learning from Bernoulli observations

Cited by in corpus (50)

• Fast calibrated additive quantile regression
• Calibrating general posterior credible regions
• Bayesian Probabilistic Numerical Methods
• Principles of Bayesian Inference using General Divergence Criteria
• Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods
• A Modern Retrospective on Probabilistic Numerics
• A comparison of learning rate selection methods in generalized Bayesian inference
• Gibbs posterior concentration rates under sub-exponential type losses
• Pseudo-Bayesian Quantum Tomography with Rank-adaptation
• Information-Theoretic Generalization Bounds for Meta-Learning and Applications
• Gibbs posterior inference on the minimum clinically important difference
• Post-Processing of MCMC
• Generalized Geographically Weighted Regression Model within a Modularized Bayesian Framework
• Model-free posterior inference on the area under the receiver operating characteristic curve
• Approximating Bayes in the 21st Century
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Adaptive Variable Selection for Sequential Prediction in Multivariate Dynamic Models
• Gibbs posterior inference on multivariate quantiles
• Random weighting in LASSO regression
• Bayesian Learning of Parameterised Quantum Circuits
• Robust and rate-optimal Gibbs posterior inference on the boundary of a noisy image
• Robust approximate Bayesian inference
• Bit Blasting Probabilistic Programs
• Generalised likelihood profiles for models with intractable likelihoods
• Robust Bayesian Regression with Synthetic Posterior
• From bilinear regression to inductive matrix completion: a quasi-Bayesian analysis
• A reduced-rank approach to predicting multiple binary responses through machine learning
• Many wrong models approach to localize an odor source in turbulence with static sensors
• Generalized Bayesian Likelihood-Free Inference
• High-dimensional sparse classification using exponential weighting with empirical hinge loss
• A loss discounting framework for model averaging and selection in time series models
• Concentration properties of fractional posterior in 1-bit matrix completion
• Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix
• Bernstein - von Mises theorem and misspecified models: a review
• Sampling algorithms in statistical physics: a guide for statistics and machine learning
• An efficient adaptive MCMC algorithm for Pseudo-Bayesian quantum tomography
• Bayesian Quantile Estimation and Regression with Martingale Posteriors
• A General Framework for Cutting Feedback within Modularised Bayesian Inference
• Computationally efficient variational-like approximations of possibilistic inferential models
• Synthetic likelihood in misspecified models
• Quantile regression for longitudinal functional data with application to feed intake of lactating sows
• Robust heavy-tailed versions of generalized linear models with applications in actuarial science
• On default priors for robust Bayesian estimation with divergences
• Generalized Bayes Approach to Inverse Problems with Model Misspecification
• Variational Gaussian Approximation in Replica Analysis of Parametric Models
• Robust posterior inference for Youden's index cutoff
• Generalized Posterior Calibration via Sequential Monte Carlo Sampler
• Modified Delayed Acceptance MCMC for Quasi-Bayesian Inference with Linear Moment Conditions
• Non-conjugate variational Bayes for pseudo-likelihood mixed effect models
• Weighted Particle-Based Optimization for Efficient Generalized Posterior Calibration