Localized Lasso for High-Dimensional Regression

Abstract

We introduce the localized Lasso, which is suited for learning models that are both interpretable and have a high predictive power in problems with high dimensionality $d$ and small sample size $n$. More specifically, we consider a function defined by local sparse models, one at each data point. We introduce sample-wise network regularization to borrow strength across the models, and sample-wise exclusive group sparsity (a.k.a., ${\ell }_{1,2}$ norm) to introduce diversity into the choice of feature sets in the local models. The local models are interpretable in terms of similarity of their sparsity patterns. The cost function is convex, and thus has a globally optimal solution. Moreover, we propose a simple yet efficient iterative least-squares based optimization procedure for the localized Lasso, which does not need a tuning parameter, and is guaranteed to converge to a globally optimal solution. The solution is empirically shown to outperform alternatives for both simulated and genomic personalized medicine data.

Cited by in corpus (10)

• Localized Linear Regression in Networked Data
• Fast local linear regression with anchor regularization
• SnapVX: A Network-Based Convex Optimization Solver
• Neural Generators of Sparse Local Linear Models for Achieving both Accuracy and Interpretability
• Adaptive Sieving with PPDNA: Generating Solution Paths of Exclusive Lasso Models
• A dual Newton based preconditioned proximal point algorithm for exclusive lasso models
• Learning Networked Exponential Families with Network Lasso
• Deep topic modeling by multilayer bootstrap network and lasso
• Distributed Networked Learning with Correlated Data
• A Bayesian approach to multi-task learning with network lasso