High-dimensional graphs and variable selection with the Lasso

Abstract

The pattern of zero entries in the inverse covariance matrix of a multivariate normal distribution corresponds to conditional independence restrictions between variables. Covariance selection aims at estimating those structural zeros from data. We show that neighborhood selection with the Lasso is a computationally attractive alternative to standard covariance selection for sparse high-dimensional graphs. Neighborhood selection estimates the conditional independence restrictions separately for each node in the graph and is hence equivalent to variable selection for Gaussian linear models. We show that the proposed neighborhood selection scheme is consistent for sparse high-dimensional graphs. Consistency hinges on the choice of the penalty parameter. The oracle value for optimal prediction does not lead to a consistent neighborhood estimate. Controlling instead the probability of falsely joining some distinct connectivity components of the graph, consistent estimation for sparse graphs is achieved (with exponential rates), even when the number of variables grows as the number of observations raised to an arbitrary power.

Published at http://dx.doi.org/10.1214/009053606000000281 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (268)

• Nearly unbiased variable selection under minimax concave penalty
• Simultaneous analysis of Lasso and Dantzig selector
• Sparse and compositionally robust inference of microbial ecological networks
• The Gaussian Graphical Model in Cross-sectional and Time-series Data
• A correlated topic model of Science
• Regularized estimation of large covariance matrices
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Boosting Algorithms: Regularization, Prediction and Model Fitting
• On the conditions used to prove oracle results for the Lasso
• Sparse permutation invariant covariance estimation
• Variable selection in nonparametric additive models
• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• High-dimensional Ising model selection using ${\ell }_1$-regularized logistic regression
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional additive modeling
• Learning graphs from data: A signal representation perspective
• Sparsistency and rates of convergence in large covariance matrix estimation
• High-dimensional generalized linear models and the lasso
• Latent variable graphical model selection via convex optimization
• Connecting the Dots: Identifying Network Structure via Graph Signal Processing
• High-dimensional variable selection
• Support union recovery in high-dimensional multivariate regression
• Estimation of high-dimensional low-rank matrices
• Estimating high-dimensional intervention effects from observational data
• Strong oracle optimality of folded concave penalized estimation
• Network exploration via the adaptive LASSO and SCAD penalties
• A Selective Review of Group Selection in High-Dimensional Models
• Near-ideal model selection by ${\ell }_1$ minimization
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Embarrassingly Shallow Autoencoders for Sparse Data
• Learning high-dimensional directed acyclic graphs with latent and selection variables
• Regularized estimation of large-scale gene association networks using graphical Gaussian models
• CAM: Causal additive models, high-dimensional order search and penalized regression
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Robust subspace clustering
• Statistical significance in high-dimensional linear models
• Least angle and ${\ell }_1$ penalized regression: A review
• L1-Penalization for Mixture Regression Models
• Signal Processing on Graphs: Causal Modeling of Unstructured Data
• Sparse recovery under matrix uncertainty
• A sparse conditional Gaussian graphical model for analysis of genetical genomics data
• Regularized rank-based estimation of high-dimensional nonparanormal graphical models
• Covariance Estimation: The GLM and Regularization Perspectives
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• Estimating time-varying networks
• SCAD-penalized regression in high-dimensional partially linear models
• Correlated variables in regression: clustering and sparse estimation
• Causal Network Inference via Group Sparse Regularization
• Multivariate Bernoulli distribution
• Regularization for Cox's proportional hazards model with NP-dimensionality
• Bayesian Structure Learning in Sparse Gaussian Graphical Models
• Coauthorship and Citation Networks for Statisticians
• Estimation for High-Dimensional Linear Mixed-Effects Models Using ${\ell }_1$-Penalization
• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Confidence intervals for high-dimensional inverse covariance estimation
• BDgraph: An R Package for Bayesian Structure Learning in Graphical Models
• High-dimensional analysis of semidefinite relaxations for sparse principal components
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Stability
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guarantees
• Some sharp performance bounds for least squares regression with $L_1$ regularization
• Feature Graph Learning for 3D Point Cloud Denoising
• High-dimensional structure estimation in Ising models: Local separation criterion
• "Pre-conditioning" for feature selection and regression in high-dimensional problems
• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• Oracle inequalities for the lasso in the Cox model
• A new perspective on least squares under convex constraint
• GLMMLasso: An Algorithm for High-Dimensional Generalized Linear Mixed Models Using L1-Penalization
• Covariance and precision matrix estimation for high-dimensional time series
• Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses
• Missing values: sparse inverse covariance estimation and an extension to sparse regression
• The sparse Laplacian shrinkage estimator for high-dimensional regression
• Robust graphical modeling of gene networks using classical and alternative T-distributions
• Network inference and biological dynamics
• Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions
• Adaptive cluster expansion for the inverse Ising problem: convergence, algorithm and tests
• Honest variable selection in linear and logistic regression models via ${\ell }_1$ and ${\ell }_1+{\ell }_2$ penalization
• Consistent group selection in high-dimensional linear regression
• Stable Graphical Model Estimation with Random Forests for Discrete, Continuous, and Mixed Variables
• Flexible covariance estimation in graphical Gaussian models
• Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap
• Parsimonious modeling with Information Filtering Networks
• Treelets--An adaptive multi-scale basis for sparse unordered data
• High-Dimensional Sparse Additive Hazards Regression
• Best subset selection, persistence in high-dimensional statistical learning and optimization under $l_1$ constraint
• Gemini: Graph estimation with matrix variate normal instances
• Sparse Principal Component based High-Dimensional Mediation Analysis
• Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
• Statistical Significance of Clustering using Soft Thresholding
• A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures
• Why Gabor Frames? Two Fundamental Measures of Coherence and Their Role in Model Selection
• Posterior convergence rates for estimating large precision matrices using graphical models
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Bayesian Gaussian Copula Graphical Modeling for Dupuytren Disease
• Sparse Regression: Scalable algorithms and empirical performance
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Networks, Dynamic Factors, and the Volatility Analysis of High-Dimensional Financial Series
• Robust Subjective Visual Property Prediction from Crowdsourced Pairwise Labels
• Change-point detection in high-dimensional covariance structure
• Factor models and variable selection in high-dimensional regression analysis
• Learning Gaussian Tree Models: Analysis of Error Exponents and Extremal Structures
• Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity
• Estimation in high-dimensional linear models with deterministic design matrices
• Network classification with applications to brain connectomics
• Change-Point Estimation in High-Dimensional Markov Random Field Models
• Gaussian graphical models reveal inter-modal and inter-regional conditional dependencies of brain alterations in Alzheimer's disease
• Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
• Goodness of fit tests for high-dimensional linear models
• UPS delivers optimal phase diagram in high-dimensional variable selection
• Time Series Analysis via Network Science: Concepts and Algorithms
• Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig
• Partial Separability and Functional Graphical Models for Multivariate Gaussian Processes
• Covariate assisted screening and estimation
• Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models
• Bootstrap inference for network construction with an application to a breast cancer microarray study
• Confidence regions for entries of a large precision matrix
• Simultaneous variable selection and estimation in semiparametric modeling of longitudinal/clustered data
• Multi-task Sparse Structure Learning
• Variable selection in functional regression models: a review
• A majorization-minimization approach to variable selection using spike and slab priors
• Learning the Conditional Independence Structure of Stationary Time Series: A Multitask Learning Approach
• Parametric or nonparametric? A parametricness index for model selection
• High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking
• Inferring gene-gene interactions and functional modules using sparse canonical correlation analysis
• Mirror averaging with sparsity priors
• Network assisted analysis to reveal the genetic basis of autism
• Inferring sparse Gaussian graphical models with latent structure
• Global solutions to folded concave penalized nonconvex learning
• Automated calibration for stability selection in penalised regression and graphical models
• Graphical Models for Zero-Inflated Single Cell Gene Expression
• SPADES and mixture models
• Inference algorithms for gene networks: a statistical mechanics analysis
• Estimating heterogeneous graphical models for discrete data with an application to roll call voting
• Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error
• Learning loopy graphical models with latent variables: Efficient methods and guarantees
• Sparsistency and agnostic inference in sparse PCA
• Discussion: Latent variable graphical model selection via convex optimization
• Stacked Penalized Logistic Regression for Selecting Views in Multi-View Learning
• Sparse Nonparametric Graphical Models
• Bayesian sparse graphical models for classification with application to protein expression data
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Introduction to correlation networks: Interdisciplinary approaches beyond thresholding
• Penalized estimation in high-dimensional hidden Markov models with state-specific graphical models
• DRAGON: Determining Regulatory Associations using Graphical models on multi-Omic Networks
• Sparse regression algorithm for activity estimation in $\gamma$ spectrometry
• The Dantzig selector: Recovery of Signal via ${\ell }_1-\alpha {\ell }_2$ Minimization
• Marginal Pseudo-Likelihood Learning of Markov Network structures
• Diffusion Adaptation Strategies for Distributed Estimation over Gaussian Markov Random Fields
• Recursive Sparse Point Process Regression with Application to Spectrotemporal Receptive Field Plasticity Analysis
• Testability of high-dimensional linear models with non-sparse structures
• Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models
• The log-linear group-lasso estimator and its asymptotic properties
• A Proximal Approach for a Class of Matrix Optimization Problems
• Estimation of Large Covariance and Precision Matrices from Temporally Dependent Observations
• Exact Blind Community Detection from Signals on Multiple Graphs
• High dimensional sparse covariance estimation via directed acyclic graphs
• Learning Gaussian Graphical Models With Fractional Marginal Pseudo-likelihood
• Separating common (global and local) and distinct variation in multiple mixed types data sets
• Quasi-Likelihood and/or Robust Estimation in High Dimensions
• Gene communities in co-expression networks across different tissues
• Identifying Network Ties from Panel Data: Theory and an Application to Tax Competition
• Blind Inference of Eigenvector Centrality Rankings
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• Estimation of Gaussian graphs by model selection
• Model Selection: Two Fundamental Measures of Coherence and Their Algorithmic Significance
• Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage
• Robust Information Criterion for Model Selection in Sparse High-Dimensional Linear Regression Models
• Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation
• Fast computation of latent correlations
• Nonlinear Causal Discovery with Confounders
• On Sparse High-Dimensional Graphical Model Learning For Dependent Time Series
• Discussion: Latent variable graphical model selection via convex optimization
• Statistical Perspective on Functional and Causal Neural Connectomics: A Comparative Study
• Asymptotic Confidence Regions for High-dimensional Structured Sparsity
• Sparse Graph Learning Under Laplacian-Related Constraints
• Sampling Requirements for Stable Autoregressive Estimation
• Exact Localization and Superresolution with Noisy Data and Random Illumination
• Rejoinder: One-step sparse estimates in nonconcave penalized likelihood models
• Joint Network Topology Inference in the Presence of Hidden Nodes
• High-Dimensional Multivariate Time Series With Additional Structure
• An information-theoretic approach to hypergraph psychometrics
• Modeling dependent gene expression
• On estimation of the diagonal elements of a sparse precision matrix
• Bayesian Inference in Nonparanormal Graphical Models
• Bayesian Inference of Networks Across Multiple Sample Groups and Data Types
• Mining Novel Multivariate Relationships in Time Series Data Using Correlation Networks
• High-dimensional variable selection via low-dimensional adaptive learning
• Joint Network Topology Inference via a Shared Graphon Model
• Model selection and local geometry
• Discussion: Latent variable graphical model selection via convex optimization
• General nonexact oracle inequalities for classes with a subexponential envelope
• Lattices of Graphical Gaussian Models with Symmetries
• Rejoinder: Latent variable graphical model selection via convex optimization
• Network reconstruction via the minimum description length principle
• A review of Gaussian Markov models for conditional independence
• High-dimensional structure learning of binary pairwise Markov networks: A comparative numerical study
• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
• Certifiably Optimal Sparse Inverse Covariance Estimation
• Predicting phenotypes from microarrays using amplified, initially marginal, eigenvector regression
• Sparse multivariate regression with missing values and its application to the prediction of material properties
• Online Structural Change-point Detection of High-dimensional Streaming Data via Dynamic Sparse Subspace Learning
• Statistical mechanics of sparse generalization and model selection
• Compressed sensing radar detectors under the row-orthogonal design model: a statistical mechanics perspective
• Joint Association Graph Screening and Decomposition for Large-scale Linear Dynamical Systems
• Modeling item--item similarities for personalized recommendations on Yahoo! front page
• View selection in multi-view stacking: Choosing the meta-learner
• Interpreting Latent Variables in Factor Models via Convex Optimization
• Cross-validation of correlation networks using modular structure
• Random Matrix Derived Shrinkage of Spectral Precision Matrices
• The Loss Rank Criterion for Variable Selection in Linear Regression Analysis
• Penalized versus constrained generalized eigenvalue problems
• A model for gene deregulation detection using expression data
• Learning High-Dimensional Differential Graphs From Multi-Attribute Data
• Learning of Tree-Structured Gaussian Graphical Models on Distributed Data under Communication Constraints
• Efficient Variational Bayes Learning of Graphical Models with Smooth Structural Changes
• High-Temperature Structure Detection in Ferromagnets
• On Learning the Optimal Regularization Parameter in Inverse Problems
• Quantum Algorithms for the Pathwise Lasso
• Learning Bayesian networks: a copula approach for mixed-type data
• Estimation of dynamic networks for high-dimensional nonstationary time series
• Adaptive estimation of the baseline hazard function in the Cox model by model selection, with high-dimensional covariates
• Aggregating Dependent Gaussian Experts in Local Approximation
• The FEDHC Bayesian network learning algorithm
• Towards the interpretation of time-varying regularization parameters in streaming penalized regression models
• Discovery and inference of a causal network with hidden confounding
• Estimation of the genetic Gaussian network using GWAS summary data
• Learning to Estimate Driver Drowsiness from Car Acceleration Sensors using Weakly Labeled Data
• Varying-coefficients for regional quantile via KNN-based LASSO with applications to health outcome study
• Large-Scale Local Causal Inference of Gene Regulatory Relationships
• Discussion: Latent variable graphical model selection via convex optimization
• High-Dimensional Screening Using Multiple Grouping of Variables
• Asymmetric Graph Error Control with Low Complexity in Causal Bandits
• Detection and inference of changes in high-dimensional linear regression with non-sparse structures
• A random covariance model for bi-level graphical modeling with application to resting-state fMRI data
• Discussion: Latent variable graphical model selection via convex optimization
• ${\ell }_1$-penalized Multinomial Regression: Estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
• Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
• Covariate-Assisted Bayesian Graph Learning for Heterogeneous Data
• A Mixture Model to Detect Edges in Sparse Co-expression Graphs
• Estimating covariance and precision matrices along subspaces
• Introduction to the Special Issue on Sparsity and Regularization Methods
• Graph Coding for Model Selection and Anomaly Detection in Gaussian Graphical Models
• Partial Correlation Graphical LASSO
• A Bayesian Graphical Approach for Large-Scale Portfolio Management with Fewer Historical Data
• A positive-definiteness-assured block Gibbs sampler for Bayesian graphical models with shrinkage priors
• Quantile mixed graphical models with an application to mass public shootings in the United States
• Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization
• On Conditional Independence Graph Learning From Multi-Attribute Gaussian Dependent Time Series
• A statistical methodology to select covariates in high-dimensional data under dependence. Application to the classification of genetic profiles in oncology
• Change point detection in dynamic Gaussian graphical models: the impact of COVID-19 pandemic on the US stock market
• Gaussian Experts Selection using Graphical Models
• Reconstructing Sparse Signals via Greedy Monte-Carlo Search
• Penalised inference for autoregressive moving average models with time-dependent predictors
• Causal Link Discovery with Unequal Edge Error Tolerance
• Online Graph Topology Learning from Matrix-valued Time Series
• Inference for Large Panel Data with Many Covariates
• Hub Detection in Gaussian Graphical Models
• A Network-Guided Penalized Regression with Application to Proteomics Data
• High-dimensional Statistical Inference and Variable Selection Using Sufficient Dimension Association
• Efficient proximal gradient algorithms for joint graphical lasso
• High-dimensional forecasting with known knowns and known unknowns
• Graphical LASSO Based Model Selection for Time Series
• KL-BSS: Rethinking optimality for neighbourhood selection in structural equation models
• Topological Techniques in Model Selection
• Ising Model Selection Using ${\ell }_1$-Regularized Linear Regression: A Statistical Mechanics Analysis
• Empowering Differential Networks Using Bayesian Analysis