Fully Bayesian Classification with Heavy-tailed Priors for Selection in High-dimensional Features with Grouping Structure
arXiv:1607.00098 · doi:10.1038/s41598-020-66466-z
Abstract
Feature selection is demanded in many modern scientific research problems that use high-dimensional data. A typical example is to find the most useful genes that are related to a certain disease (eg, cancer) from high-dimensional gene expressions. The expressions of genes have grouping structures, for example, a group of co-regulated genes that have similar biological functions tend to have similar expressions. Many statistical methods have been proposed to take the grouping structure into consideration in feature selection, including group LASSO, supervised group LASSO, and regression on group representatives. In this paper, we propose a fully Bayesian Robit regression method with heavy-tailed (sparsity) priors (shortened by FBRHT) for selecting features with grouping structure. The main features of FBRHT include that it discards more aggressively unrelated features than LASSO, and it can make feature selection within groups automatically without a pre-specified grouping structure. In this paper, we use simulated and real datasets to demonstrate that the predictive power of the sparse feature subsets selected by FBRHT are comparable with other much larger feature subsets selected by LASSO, group LASSO, supervised group LASSO, penalized logistic regression and random forest, and that the succinct feature subsets selected by FBRHT have significantly better predictive power than the feature subsets of the same size taken from the top features selected by the aforementioned methods.
arXiv comment31 pages
References in corpus (16)
- XGBoost: A Scalable Tree Boosting System
- In silico prediction of protein-protein interactions in human macrophages
- Nearly unbiased variable selection under minimax concave penalty
- MCMC using Hamiltonian dynamics
- A weakly informative default prior distribution for logistic and other regression models
- Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- Comparison of Bayesian predictive methods for model selection
- Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
- Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima
- Gene Hunting with Knockoffs for Hidden Markov Models
- The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- Fully Bayesian Logistic Regression with Hyper-Lasso Priors for High-dimensional Feature Selection
- Good, great, or lucky? Screening for firms with sustained superior performance using heavy-tailed priors
- Classification of arrayCGH data using a fused SVM