On the conditions used to prove oracle results for the Lasso

Abstract

Oracle inequalities and variable selection properties for the Lasso in linear models have been established under a variety of different assumptions on the design matrix. We show in this paper how the different conditions and concepts relate to each other. The restricted eigenvalue condition (Bickel et al., 2009) or the slightly weaker compatibility condition (van de Geer, 2007) are sufficient for oracle results. We argue that both these conditions allow for a fairly general class of design matrices. Hence, optimality of the Lasso for prediction and estimation holds for more general situations than what it appears from coherence (Bunea et al, 2007b,c) or restricted isometry (Candes and Tao, 2005) assumptions.

33 pages, 1 figure

References in corpus (9)

• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional generalized linear models and the lasso
• Near-ideal model selection by ${\ell }_1$ minimization
• Sparsity oracle inequalities for the Lasso
• Aggregation for Gaussian regression
• The Dantzig selector and sparsity oracle inequalities
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Some sharp performance bounds for least squares regression with $L_1$ regularization

Cited by in corpus (199)

• The Convex Geometry of Linear Inverse Problems
• On asymptotically optimal confidence regions and tests for high-dimensional models
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• Regularized estimation in sparse high-dimensional time series models
• Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations
• Strong oracle optimality of folded concave penalized estimation
• A Selective Review of Group Selection in High-Dimensional Models
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Bayesian linear regression with sparse priors
• Asymptotic normality and optimalities in estimation of large Gaussian graphical models
• Statistical significance in high-dimensional linear models
• L1-Penalization for Mixture Regression Models
• High-Dimensional Inference: Confidence Intervals, $p$-Values and R-Software hdi
• On the Prediction Performance of the Lasso
• Correlated variables in regression: clustering and sparse estimation
• Causal Network Inference via Group Sparse Regularization
• Regularization for Cox's proportional hazards model with NP-dimensionality
• Estimation for High-Dimensional Linear Mixed-Effects Models Using ${\ell }_1$-Penalization
• Pivotal estimation via square-root Lasso in nonparametric regression
• Multi-Stage Multi-Task Feature Learning
• Confidence intervals for high-dimensional inverse covariance estimation
• The Lasso for High-Dimensional Regression with a Possible Change-Point
• Oracle inequalities for the lasso in the Cox model
• Lower bounds on the performance of polynomial-time algorithms for sparse linear regression
• Exponential Screening and optimal rates of sparse estimation
• Sparse Matrix Inversion with Scaled Lasso
• Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression
• The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning
• A Practical Scheme and Fast Algorithm to Tune the Lasso With Optimality Guarantees
• Change-Point Estimation in High-Dimensional Markov Random Field Models
• Goodness of fit tests for high-dimensional linear models
• Efficient Sparse Group Feature Selection via Nonconvex Optimization
• The Multivariate Hawkes Process in High Dimensions: Beyond Mutual Excitation
• Network Granger Causality with Inherent Grouping Structure
• Thresholded Lasso for high dimensional variable selection and statistical estimation
• Selective Sequential Model Selection
• Union Support Recovery in Multi-task Learning
• Vector-Valued Graph Trend Filtering with Non-Convex Penalties
• Local stability and robustness of sparse dictionary learning in the presence of noise
• Bayesian and L1 Approaches to Sparse Unsupervised Learning
• Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error
• Sparsity considerations for dependent observations
• Pac-bayesian bounds for sparse regression estimation with exponential weights
• Adaptive Lasso and group-Lasso for functional Poisson regression
• Inferring Graphs from Cascades: A Sparse Recovery Framework
• Variable Selection is Hard
• Estimating the error variance in a high-dimensional linear model
• High-Dimensional Estimation of Structured Signals from Non-Linear Observations with General Convex Loss Functions
• Sparse regression algorithm for activity estimation in $\gamma$ spectrometry
• Learning Local Dependence In Ordered Data
• DIF Statistical Inference without Knowing Anchoring Items
• Statistically and Computationally Efficient Change Point Localization in Regression Settings
• Doubly-Robust Lasso Bandit
• Trust, but verify: benefits and pitfalls of least-squares refitting in high dimensions
• Accuracy guarantees for L1-recovery
• Asymptotic Analysis of LASSOs Solution Path with Implications for Approximate Message Passing
• Localizing Changes in High-Dimensional Vector Autoregressive Processes
• Atomic norm denoising with applications to line spectral estimation
• Quasi-Likelihood and/or Robust Estimation in High Dimensions
• The generalized Lasso with non-linear observations
• Scaled Sparse Linear Regression
• Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications
• Impacts of high dimensionality in finite samples
• On the Post Selection Inference constant under Restricted Isometry Properties
• Model Selection for High-Dimensional Regression under the Generalized Irrepresentability Condition
• Group sparse optimization via ${\ell }_{p,q}$ regularization
• Sparsity regret bounds for individual sequences in online linear regression
• Compressed Sensing over ${\ell }_p$-balls: Minimax Mean Square Error
• Minimax Optimal Sparse Signal Recovery with Poisson Statistics
• Confidence Intervals for Low-Dimensional Parameters in High-Dimensional Linear Models
• Bayesian Sparse Linear Regression with Unknown Symmetric Error
• Accuracy guaranties for ${\ell }_1$ recovery of block-sparse signals
• Combined l_1 and greedy l_0 penalized least squares for linear model selection
• A General Framework of Dual Certificate Analysis for Structured Sparse Recovery Problems
• Robust subset selection
• Sparse High-Dimensional Linear Regression. Algorithmic Barriers and a Local Search Algorithm
• SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax
• Structured Sparse Regression via Greedy Hard-Thresholding
• The adaptive and the thresholded Lasso for potentially misspecified models
• Multiple-Splitting Projection Test for High-Dimensional Mean Vectors
• Fast global convergence of gradient methods for high-dimensional statistical recovery
• Sparse Signal Recovery under Poisson Statistics
• An l1-Oracle Inequality for the Lasso
• Adaptive Minimax Estimation over Sparse ${\ell }_q$-Hulls
• The Group Square-Root Lasso: Theoretical Properties and Fast Algorithms
• Randomized maximum-contrast selection: subagging for large-scale regression
• Selective Factor Extraction in High Dimensions
• False Discovery Rate Control via Data Splitting
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• A review of Gaussian Markov models for conditional independence
• I-LAMM for Sparse Learning: Simultaneous Control of Algorithmic Complexity and Statistical Error
• Inference for high-dimensional instrumental variables regression
• Localizing Changes in High-Dimensional Regression Models
• Nonparametric Regression with Adaptive Truncation via a Convex Hierarchical Penalty
• A study on tuning parameter selection for the high-dimensional lasso
• Active Learning Algorithms for Graphical Model Selection
• Network Reconstruction and Prediction of Epidemic Outbreaks for NIMFA Processes
• Generalized Kalman Smoothing: Modeling and Algorithms
• Discussion: "A significance test for the lasso"
• Variable Selection with ABC Bayesian Forests
• Joint Estimation and Inference for Data Integration Problems based on Multiple Multi-layered Gaussian Graphical Models
• Regression modeling on stratified data with the lasso
• Joint estimation of $K$ related regression models with simple $L_1$-norm penalties
• The Statistics of Streaming Sparse Regression
• False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation
• Neuronized Priors for Bayesian Sparse Linear Regression
• CoCoLasso for High-dimensional Error-in-variables Regression
• Uniform Inference in High-dimensional Dynamic Panel Data Models
• Prior Adaptive Semi-supervised Learning with Application to EHR Phenotyping
• Adaptive estimation of the baseline hazard function in the Cox model by model selection, with high-dimensional covariates
• An analysis of penalized interaction models
• Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction
• Finite mixture regression: A sparse variable selection by model selection for clustering
• High-dimensional semi-supervised learning: in search for optimal inference of the mean
• Decomposable Norm Minimization with Proximal-Gradient Homotopy Algorithm
• Link Prediction in Graphs with Autoregressive Features
• Estimating the Lasso's Effective Noise
• Sparsity-Agnostic Lasso Bandit
• CoxKnockoff: Controlled Feature Selection for the Cox Model Using Knockoffs
• High-Dimensional Confidence Regions in Sparse MRI
• Linear convergence of SDCA in statistical estimation
• A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems
• The Smooth-Lasso and other ${\ell }_1+{\ell }_2$-penalized methods
• High dimensional regression and matrix estimation without tuning parameters
• REMI: Regression with marginal information and its application in genome-wide association studies
• Semiparametric Sparse Discriminant Analysis
• Sparse and spurious: dictionary learning with noise and outliers
• Graph-based regularization for regression problems with alignment and highly-correlated designs
• Regularity Properties for Sparse Regression
• Spatially-adaptive sensing in nonparametric regression
• Off-the-grid learning of mixtures from a continuous dictionary
• Gaining Outlier Resistance with Progressive Quantiles: Fast Algorithms and Theoretical Studies
• Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• A Theoretical Analysis of Sparse Recovery Stability of Dantzig Selector and LASSO
• Minimax optimal convex methods for Poisson inverse problems under ${\ell }_q$-ball sparsity
• Sparse Partially Linear Additive Models
• Online Sparse Reinforcement Learning
• Sparse Group Selection Through Co-Adaptive Penalties
• Sharp Support Recovery from Noisy Random Measurements by L1 minimization
• Markov Neighborhood Regression for High-Dimensional Inference
• Uncertainty Quantification Under Group Sparsity
• Distributed Sparse Regression via Penalization
• Sparse Recovery from Extreme Eigenvalues Deviation Inequalities
• The $l_q$ consistency of the Dantzig Selector for Cox's Proportional Hazards Model
• Submodularity in Statistics: Comparing the Success of Model Selection Methods
• Indirect Gaussian Graph Learning beyond Gaussianity
• On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions
• Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators
• Calibrated zero-norm regularized LS estimator for high-dimensional error-in-variables regression
• High-dimensional Adaptive Minimax Sparse Estimation with Interactions
• M-estimation with the Trimmed l1 Penalty
• Extreme Eigenvalues of Nonlinear Correlation Matrices with Applications to Additive Models
• Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees
• Outlier-robust sparse/low-rank least-squares regression and robust matrix completion
• Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression
• Topologically penalized regression on manifolds
• Inferring serial correlation with dynamic backgrounds
• Oracle inequalities for sign constrained generalized linear models
• Sorted Concave Penalized Regression
• Sparse and Robust Linear Regression: An Optimization Algorithm and Its Statistical Properties
• Structural Change in Sparsity
• Sparse regression and marginal testing using cluster prototypes
• Constraints and Conditions: the Lasso Oracle-inequalities
• Efficient Clustering of Correlated Variables and Variable Selection in High-Dimensional Linear Models
• Directing Power Towards Conic Parameter Subspaces
• Learning Gaussian DAGs from Network Data
• On prediction with the LASSO when the design is not incoherent
• KL-BSS: Rethinking optimality for neighbourhood selection in structural equation models
• Inference Without Compatibility
• Dantzig Selector with an Approximately Optimal Denoising Matrix and its Application to Reinforcement Learning
• Sparse Multivariate ARCH Models: Finite Sample Properties
• Adaptive Estimation In High-Dimensional Additive Models With Multi-Resolution Group Lasso
• Robust Lasso with missing and grossly corrupted observations
• Logistic regression and Ising networks: prediction and estimation when violating lasso assumptions
• Lasso-type estimators for Semiparametric Nonlinear Mixed-Effects Models Estimation
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors
• Predicting sparse circle maps from their dynamics
• High-dimensional regression with unknown variance
• New Error Analysis for Lasso
• Graphical LASSO Based Model Selection for Time Series
• Total Variation Regularized Tensor-on-scalar Regression
• A Simple Homotopy Proximal Mapping for Compressive Sensing
• A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem
• Multi-stage Multi-task feature learning via adaptive threshold
• Robust Elastic Net Regression
• An \ell_1-oracle inequality for the Lasso in finite mixture of multivariate Gaussian regression models
• Generalized Matrix Decomposition Regression: Estimation and Inference for Two-way Structured Data
• Multi-task Learning with High-Dimensional Noisy Images
• A Global Homogeneity Test for High-Dimensional Linear Regression
• On the uniform convergence of empirical norms and inner products, with application to causal inference
• On sure early selection of the best subset
• Fine-Gray competing risks model with high-dimensional covariates: estimation and Inference
• High Dimensional Logistic Regression Under Network Dependence
• Best Subset Selection in Reduced Rank Regression
• A proximal dual semismooth Newton method for computing zero-norm penalized QR estimator