Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models

Abstract

A challenging problem in estimating high-dimensional graphical models is to choose the regularization parameter in a data-dependent way. The standard techniques include $K$-fold cross-validation ($K$-CV), Akaike information criterion (AIC), and Bayesian information criterion (BIC). Though these methods work well for low-dimensional problems, they are not suitable in high dimensional settings. In this paper, we present StARS: a new stability-based method for choosing the regularization parameter in high dimensional inference for undirected graphs. The method has a clear interpretation: we use the least amount of regularization that simultaneously makes a graph sparse and replicable under random sampling. This interpretation requires essentially no conditions. Under mild conditions, we show that StARS is partially sparsistent in terms of graph estimation: i.e. with high probability, all the true edges will be included in the selected model even when the graph size diverges with the sample size. Empirically, the performance of StARS is compared with the state-of-the-art model selection procedures, including $K$-CV, AIC, and BIC, on both synthetic data and a real microarray dataset. StARS outperforms all these competing procedures.

References in corpus (2)

• Model Selection Through Sparse Maximum Likelihood Estimation
• Stability Selection

Cited by in corpus (26)

• Controlling the false discovery rate via knockoffs
• BDgraph: An R Package for Bayesian Structure Learning in Graphical Models
• On Graphical Models via Univariate Exponential Family Distributions
• High-Dimensional Gaussian Graphical Model Selection: Walk Summability and Local Separation Criterion
• Bayesian model choice and information criteria in sparse generalized linear models
• The Nonparanormal SKEPTIC
• Local and Global Inference for High Dimensional Nonparanormal Graphical Models
• Two Sample Inference for Populations of Graphical Models with Applications to Functional Connectivity
• Fused Multiple Graphical Lasso
• A General Framework for Mixed Graphical Models
• Conic Optimization Theory: Convexification Techniques and Numerical Algorithms
• Sparse Median Graphs Estimation in a High Dimensional Semiparametric Model
• Efficient Regularized Regression for Variable Selection with L0 Penalty
• A Junction Tree Framework for Undirected Graphical Model Selection
• Joint Estimation of Multiple Graphical Models from High Dimensional Time Series
• On the Impact of Dimension Reduction on Graphical Structures
• Dependence Modeling in Ultra High Dimensions with Vine Copulas and the Graphical Lasso
• Streaming regularization parameter selection via stochastic gradient descent
• Bayesian Modeling of Multiple Structural Connectivity Networks During the Progression of Alzheimer's Disease
• Adaptive regularization for Lasso models in the context of non-stationary data streams
• A Log-Linear Graphical Model for Inferring Genetic Networks from High-Throughput Sequencing Data
• Model Selection With Graphical Neighbour Information
• Tree-based Inference of Species Interaction Network from Abundance Data
• Penalized polytomous ordinal logistic regression using cumulative logits. Application to network inference of zero-inflated variables
• Detection of Gene-Gene Interactions by Multistage Sparse and Low-Rank Regression
• A computationally fast alternative to cross-validation in penalized Gaussian graphical models