Projection Predictive Inference for Generalized Linear and Additive Multilevel Models

Abstract

Projection predictive inference is a decision theoretic Bayesian approach that decouples model estimation from decision making. Given a reference model previously built including all variables present in the data, projection predictive inference projects its posterior onto a constrained space of a subset of variables. Variable selection is then performed by sequentially adding relevant variables until predictive performance is satisfactory. Previously, projection predictive inference has been demonstrated only for generalized linear models (GLMs) and Gaussian processes (GPs) where it showed superior performance to competing variable selection procedures. In this work, we extend projection predictive inference to support variable and structure selection for generalized linear multilevel models (GLMMs) and generalized additive multilevel models (GAMMs). Our simulative and real-word experiments demonstrate that our method can drastically reduce the model complexity required to reach reference predictive performance and achieve good frequency properties.

References in corpus (6)

• Spike and slab variable selection: Frequentist and Bayesian strategies
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Comparison of Bayesian predictive methods for model selection
• Bayesian variable selection with shrinking and diffusing priors
• Projective Inference in High-dimensional Problems: Prediction and Feature Selection
• Projection predictive model selection for Gaussian processes

Cited by in corpus (6)

• Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Latent space projection predictive inference
• Generalized Decomposition Priors on R2
• On Reparameterization Invariant Bayesian Point Estimates and Credible Regions
• Bayesian clustering using random effects models and predictive projections