Projection predictive variable selection for discrete response families with finite support

Abstract

The projection predictive variable selection is a decision-theoretically justified Bayesian variable selection approach achieving an outstanding trade-off between predictive performance and sparsity. Its projection problem is not easy to solve in general because it is based on the Kullback-Leibler divergence from a restricted posterior predictive distribution of the so-called reference model to the parameter-conditional predictive distribution of a candidate model. Previous work showed how this projection problem can be solved for response families employed in generalized linear models and how an approximate latent-space approach can be used for many other response families. Here, we present an exact projection method for all response families with discrete and finite support, called the augmented-data projection. A simulation study for an ordinal response family shows that the proposed method performs better than or similarly to the previously proposed approximate latent-space projection. The cost of the slightly better performance of the augmented-data projection is a substantial increase in runtime. Thus, in such cases, we recommend the latent projection in the early phase of a model-building workflow and the augmented-data projection for final results. The ordinal response family from our simulation study is supported by both projection methods, but we also include a real-world cancer subtyping example with a nominal response family, a case that is not supported by the latent projection.

This preprint has not undergone peer review (when applicable) or any post-submission improvements or corrections. The Version of Record of this article is published in Computational Statistics, and is available online at https://doi.org/10.1007/s00180-024-01506-0

References in corpus (5)

• A weakly informative default prior distribution for logistic and other regression models
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• BayesSUR: An R package for high-dimensional multivariate Bayesian variable and covariance selection in linear regression
• Latent space projection predictive inference
• Advances in projection predictive inference

Cited by in corpus (1)

• Efficient estimation and correction of selection-induced bias with order statistics