The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties

Abstract

We study the estimation capacity of the generalized Lasso, i.e., least squares minimization combined with a (convex) structural constraint. While Lasso-type estimators were originally designed for noisy linear regression problems, it has recently turned out that they are in fact robust against various types of model uncertainties and misspecifications, most notably, non-linearly distorted observation models. This work provides more theoretical evidence for this somewhat astonishing phenomenon. At the heart of our analysis stands the mismatch principle, which is a simple recipe to establish theoretical error bounds for the generalized Lasso. The associated estimation guarantees are of independent interest and are formulated in a fairly general setup, permitting arbitrary sub-Gaussian data, possibly with strongly correlated feature designs; in particular, we do not assume a specific observation model which connects the input and output variables. Although the mismatch principle is conceived based on ideas from statistical learning theory, its actual application area are (high-dimensional) estimation tasks for semi-parametric models. In this context, the benefits of the mismatch principle are demonstrated for a variety of popular problem classes, such as single-index models, generalized linear models, and variable selection. Apart from that, our findings are also relevant to recent advances in quantized and distributed compressed sensing.

References in corpus (7)

• The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• On the Connection Between Learning Two-Layers Neural Networks and Tensor Decomposition
• On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models
• Lifting high-dimensional nonlinear models with Gaussian regressors
• Graph-based regularization for regression problems with alignment and highly-correlated designs
• A convex formulation for high-dimensional sparse sliced inverse regression

Cited by in corpus (5)

• The Generalized Lasso with Nonlinear Observations and Generative Priors
• Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data
• A Unified Approach to Uniform Signal Recovery From Non-Linear Observations
• The Restricted Isometry of ReLU Networks: Generalization through Norm Concentration
• Quickly Finding the Best Linear Model in High Dimensions