The LASSO risk for gaussian matrices

Abstract

We consider the problem of learning a coefficient vector x_0\in R^N from noisy linear observation y=Ax_0+w \in R^n. In many contexts (ranging from model selection to image processing) it is desirable to construct a sparse estimator x'. In this case, a popular approach consists in solving an L1-penalized least squares problem known as the LASSO or Basis Pursuit DeNoising (BPDN). For sequences of matrices A of increasing dimensions, with independent gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic mean square error of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical models ideas. Simulations on real data matrices suggest that our results can be relevant in a broad array of practical applications.

43 pages, 5 figures (v3 rectifies some inconsistencies in the formulation of auxiliary lemmas)

Cited by in corpus (7)

• A framework to characterize performance of LASSO algorithms
• Compressed Sensing over ${\ell }_p$-balls: Minimax Mean Square Error
• Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing
• Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising
• A performance analysis framework for SOCP algorithms in noisy compressed sensing
• A problem dependent analysis of SOCP algorithms in noisy compressed sensing
• The Sampling Rate-Distortion Tradeoff for Sparsity Pattern Recovery in Compressed Sensing