A weakly informative default prior distribution for logistic and other regression models

Abstract

We propose a new prior distribution for classical (nonhierarchical) logistic regression models, constructed by first scaling all nonbinary variables to have mean 0 and standard deviation 0.5, and then placing independent Student-$t$ prior distributions on the coefficients. As a default choice, we recommend the Cauchy distribution with center 0 and scale 2.5, which in the simplest setting is a longer-tailed version of the distribution attained by assuming one-half additional success and one-half additional failure in a logistic regression. Cross-validation on a corpus of datasets shows the Cauchy class of prior distributions to outperform existing implementations of Gaussian and Laplace priors. We recommend this prior distribution as a default choice for routine applied use. It has the advantage of always giving answers, even when there is complete separation in logistic regression (a common problem, even when the sample size is large and the number of predictors is small), and also automatically applying more shrinkage to higher-order interactions. This can be useful in routine data analysis as well as in automated procedures such as chained equations for missing-data imputation. We implement a procedure to fit generalized linear models in R with the Student-$t$ prior distribution by incorporating an approximate EM algorithm into the usual iteratively weighted least squares. We illustrate with several applications, including a series of logistic regressions predicting voting preferences, a small bioassay experiment, and an imputation model for a public health data set.

Published in at http://dx.doi.org/10.1214/08-AOAS191 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (68)

• Visualization in Bayesian workflow
• Philosophy and the practice of Bayesian statistics
• Estimating treatment effect heterogeneity in randomized program evaluation
• The prior can generally only be understood in the context of the likelihood
• Firth's logistic regression with rare events: accurate effect estimates AND predictions?
• Statistical methods for linguistic research: Foundational Ideas - Part II
• On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis
• Meta-analysis of few studies involving rare events
• Bayesian comparison of latent variable models: Conditional vs marginal likelihoods
• Theory of self-learning $Q$-matrix
• Variable Selection for Latent Class Analysis with Application to Low Back Pain Diagnosis
• Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution
• Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation
• Conjugate Bayes for probit regression via unified skew-normal distributions
• Bayes, Jeffreys, Prior Distributions and the Philosophy of Statistics
• Weak Informativity and the Information in One Prior Relative to Another
• Bayesian Nonparametric Weighted Sampling Inference
• The Dynamics of Emotions in Online Interaction
• Evaluation of Logistic Regression Applied to Respondent-Driven Samples: Simulated and Real Data
• The Overlooked Potential of Generalized Linear Models in Astronomy - I: Binomial Regression
• Spectral likelihood expansions for Bayesian inference
• Probability aggregation in time-series: Dynamic hierarchical modeling of sparse expert beliefs
• Inference of compressed Potts graphical models
• Multivariate Bayesian Logistic Regression for Analysis of Clinical Study Safety Issues
• The seven sisters DANCe IV. Bayesian hierarchical model
• Confronting Quasi-Separation in Logistic Mixed Effects for Linguistic Data: A Bayesian Approach
• Sparsity-Exploiting Anchor Placement for Localization in Sensor Networks
• Low-Cost Energy Meter Calibration Method for Measurement and Verification
• Holes in Bayesian Statistics
• Optimal multi-wave sampling for regression modelling in two-phase designs
• MUCE: Bayesian Hierarchical Modeling for the Design and Analysis of Phase 1b Multiple Expansion Cohort Trials
• Fully Bayesian Logistic Regression with Hyper-Lasso Priors for High-dimensional Feature Selection
• Fully Bayesian Classification with Heavy-tailed Priors for Selection in High-dimensional Features with Grouping Structure
• Bayesian hierarchical inference of asteroseismic inclination angles
• Rebar: Reinforcing a Matching Estimator with Predictions from High-Dimensional Covariates
• Hierarchical Bayesian CMB Component Separation with the No-U-Turn Sampler
• Modeling the effects of telephone nursing on healthcare utilization
• Probabilistic Model Updating of Steel Frame Structures Using Strain and Acceleration Measurements: A Multitask Learning Framework
• Better Classifier Calibration for Small Data Sets
• Bayes-raking: Bayesian Finite Population Inference with Known Margins
• Highly Scalable Maximum Likelihood and Conjugate Bayesian Inference for ERGMs on Graph Sets with Equivalent Vertices
• Bayesian multilevel multivariate logistic regression for superiority decision-making under observable treatment heterogeneity
• A closed-form filter for binary time series
• A Bayesian accelerated failure time model for interval censored three-state screening outcomes
• A hierarchical Bayesian model to infer PL(Z) relations using Gaia parallaxes
• Bayesian design and analysis of external pilot trials for complex interventions
• Random-effects meta-analysis of phase I dose-finding studies using stochastic process priors
• ROMI: A Randomized Two-Stage Basket Trial Design to Optimize Doses for Multiple Indications
• Evaluating hybrid controls methodology in early-phase oncology trials: a simulation study based on the MORPHEUS-UC trial
• Amplitude analysis of $B^{+}\to \psi (2S)K^{+}{\pi }^{+}{\pi }^{-}$ decays
• Informative Bayesian model selection for RR Lyrae star classifiers
• Projection predictive variable selection for discrete response families with finite support
• Analyzing Population-Level Trials as N-of-1 Trials: an Application to Gait
• Inference of a universal social scale and segregation measures using social connectivity kernels
• P-splines with an l1 penalty for repeated measures
• A Bayesian hierarchical model for bridging across patient subgroups in phase I clinical trials with animal data
• How much spin wandering can continuous gravitational wave search algorithms handle?
• A Bayesian approach to functional regression: theory and computation
• Unsupervised Bayesian classification for models with scalar and functional covariates
• Persistent disruption of interspecific competition after ultra-low esfenvalerate exposure
• Doubly Adaptive Importance Sampling
• Eliciting prior information from clinical trials via calibrated Bayes factor
• Bayesian random-effects meta-analysis of aggregate data on clinical events
• Bayesian generalized method of moments applied to pseudo-observations in survival analysis
• Scalable couplings for the random walk Metropolis algorithm
• Bayesian multivariate logistic regression for superiority and inferiority decision-making under observable treatment heterogeneity
• Bayesian Unbiasing of the Gaia Space Mission Time Series Database
• Public Domain Rank: Identifying Notable Individuals with the Wisdom of the Crowd