Trace Lasso: a trace norm regularization for correlated designs

Abstract

Using the ${\ell }_1$-norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, called the trace Lasso, uses the trace norm, which is a convex surrogate of the rank, of the selected covariates as the criterion of model complexity. We analyze the properties of our norm, describe an optimization algorithm based on reweighted least-squares, and illustrate the behavior of this norm on synthetic data, showing that it is more adapted to strong correlations than competing methods such as the elastic net.

References in corpus (2)

• Consistency of the group Lasso and multiple kernel learning
• Consistency of trace norm minimization

Cited by in corpus (16)

• Robust Face Recognition via Adaptive Sparse Representation
• Small-sample Brain Mapping: Sparse Recovery on Spatially Correlated Designs with Randomization and Clustering
• Local Linear Convergence of Forward-Backward under Partial Smoothness
• Convex Learning of Multiple Tasks and their Structure
• The generalized Lasso with non-linear observations
• A Generalized Online Mirror Descent with Applications to Classification and Regression
• Swapping Variables for High-Dimensional Sparse Regression with Correlated Measurements
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction
• SuRF: a New Method for Sparse Variable Selection, with Application in Microbiome Data Analysis
• Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
• Support recovery and sup-norm convergence rates for sparse pivotal estimation
• CAST: A Correlation-based Adaptive Spectral Clustering Algorithm on Multi-scale Data
• Bayesian Inference with the l1-ball Prior: Solving Combinatorial Problems with Exact Zeros
• Spectral Clustering with Smooth Tiny Clusters
• Diverse Online Feature Selection
• An Inexact Manifold Augmented Lagrangian Method for Adaptive Sparse Canonical Correlation Analysis with Trace Lasso Regularization