A General Framework for the Parametrization of Hierarchical Models

Abstract

In this paper, we describe centering and noncentering methodology as complementary techniques for use in parametrization of broad classes of hierarchical models, with a view to the construction of effective MCMC algorithms for exploring posterior distributions from these models. We give a clear qualitative understanding as to when centering and noncentering work well, and introduce theory concerning the convergence time complexity of Gibbs samplers using centered and noncentered parametrizations. We give general recipes for the construction of noncentered parametrizations, including an auxiliary variable technique called the state-space expansion technique. We also describe partially noncentered methods, and demonstrate their use in constructing robust Gibbs sampler algorithms whose convergence properties are not overly sensitive to the data.

Published at http://dx.doi.org/10.1214/088342307000000014 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (2)

• Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical model
• Stability of the Gibbs Sampler for Bayesian Hierarchical Models

Cited by in corpus (70)

• Bayesian Compression for Deep Learning
• Statistical methods for linguistic research: Foundational Ideas - Part II
• Validating Bayesian Inference Algorithms with Simulation-Based Calibration
• Nested Sampling Methods
• Statistical methods for linguistic research: Foundational Ideas - Part I
• Efficient Bayesian Inference for Multivariate Factor Stochastic Volatility Models
• Parameterizations for Ensemble Kalman Inversion
• Monte Carlo Gradient Estimation in Machine Learning
• Diagnosing Suboptimal Cotangent Disintegrations in Hamiltonian Monte Carlo
• MCMC-Based Inference in the Era of Big Data: A Fundamental Analysis of the Convergence Complexity of High-Dimensional Chains
• Variable transformation to obtain geometric ergodicity in the random-walk Metropolis algorithm
• MUSE: Marginal Unbiased Score Expansion and Application to CMB Lensing
• Bayesian inference of CMB gravitational lensing
• Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets
• Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations
• Probabilistic prediction of neurological disorders with a statistical assessment of neuroimaging data modalities
• Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors
• Metropolized Randomized Maximum Likelihood for sampling from multimodal distributions
• Convergence analysis of the Gibbs sampler for Bayesian general linear mixed models with improper priors
• Improving the Convergence Properties of the Data Augmentation Algorithm with an Application to Bayesian Mixture Modeling
• Variational approximation for mixtures of linear mixed models
• Dimension-Robust MCMC in Bayesian Inverse Problems
• Coupling stochastic EM and Approximate Bayesian Computation for parameter inference in state-space models
• Bayesian computation: a perspective on the current state, and sampling backwards and forwards
• Machine Learning Meets Microeconomics: The Case of Decision Trees and Discrete Choice
• Enhanced monitoring of atmospheric methane from space over the Permian basin with hierarchical Bayesian inference
• Efficient Gravitational Wave Searches with Pulsar Timing Arrays using Hamiltonian Monte Carlo
• Accelerating Parallel Tempering: Quantile Tempering Algorithm (QuanTA)
• Big Learning with Bayesian Methods
• A stochastic variational framework for fitting and diagnosing generalized linear mixed models
• A non-parametric Bayesian approach to decompounding from high frequency data
• A scalable optimal-transport based local particle filter
• Scalable Bayesian computation for crossed and nested hierarchical models
• Reducing nuisance prior sensitivity via non-linear reparameterization, with application to EFT analyses of large-scale structure
• Analysis of Hierarchical Ensemble Kalman Inversion
• Efficient parameter sampling for Markov jump processes
• Partially factorized variational inference for high-dimensional mixed models
• Scalable iterative methods for sampling from massive Gaussian random vectors
• Automatic Reparameterisation of Probabilistic Programs
• A linear noise approximation for stochastic epidemic models fit to partially observed incidence counts
• Informative Bayesian Neural Network Priors for Weak Signals
• How Deep Are Deep Gaussian Processes?
• Adaptive inference over Besov spaces in the white noise model using $p$-exponential priors
• Sampling algorithms in statistical physics: a guide for statistics and machine learning
• Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations
• Data transforming augmentation for heteroscedastic models
• Improved Gibbs samplers for Cosmic Microwave Background power spectrum estimation
• Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions
• Bayesian prediction of jumps in large panels of time series data
• Linear Mixed-Effects Models for Non-Gaussian Repeated Measurement Data
• Latent diffusion models for survival analysis
• Bayesian Estimation of the Degrees of Freedom Parameter of the Student-$t$ Distribution---A Beneficial Re-parameterization
• A Conversation with Alan Gelfand
• Rapid inference for individual binaries and a stochastic background with pulsar timing array data
• Posterior-based proposals for speeding up Markov chain Monte Carlo
• Gaussian distributional structural equation models: A framework for modeling latent heteroscedasticity
• Variationally Inferred Sampling Through a Refined Bound for Probabilistic Programs
• Geometric Conditions for the Discrepant Posterior Phenomenon and Connections to Simpson's Paradox
• Analysis of the Gibbs sampler for hierarchical inverse problems
• Bayesian Inference for Gaussian Process Classifiers with Annealing and Pseudo-Marginal MCMC
• Pseudo-Marginal Bayesian Inference for Gaussian Processes
• Markov chain Monte Carlo for exact inference for diffusions
• Unfolding the Network of Peer Grades: A Latent Variable Approach
• FunMC: A functional API for building Markov Chains
• Scattering and sputtering on the lunar surface; Insights from negative ions observed at the surface
• Sampling hyperparameters in hierarchical models: improving on Gibbs for high-dimensional latent fields and large data sets
• Benchmarking field-level cosmological inference from galaxy redshift surveys
• Incomplete Reparameterizations and Equivalent Metrics
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
• Multilevel linear models, Gibbs samplers and multigrid decompositions