Multi-Task Feature Learning Via Efficient l2,1-Norm Minimization
arXiv:1205.2631
Abstract
The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the l2,1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assuming a suitable prior from the exponential family. One appealing feature of the l2,1-norm regularization is that it encourages multiple predictors to share similar sparsity patterns. However, the resulting optimization problem is challenging to solve due to the non-smoothness of the l2,1-norm regularization. In this paper, we propose to accelerate the computation by reformulating it as two equivalent smooth convex optimization problems which are then solved via the Nesterov's method-an optimal first-order black-box method for smooth convex optimization. A key building block in solving the reformulations is the Euclidean projection. We show that the Euclidean projection for the first reformulation can be analytically computed, while the Euclidean projection for the second one can be computed in linear time. Empirical evaluations on several data sets verify the efficiency of the proposed algorithms.
Appears in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI2009)
References in corpus (1)
Cited by in corpus (40)
- A survey of sparse representation: algorithms and applications
- Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation
- Multi-Task Learning for Dense Prediction Tasks: A Survey
- Group Sparse Regularization for Deep Neural Networks
- Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
- TopologyNet: Topology based deep convolutional neural networks for biomolecular property predictions
- Learning Task Grouping and Overlap in Multi-task Learning
- Smoothing proximal gradient method for general structured sparse regression
- Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
- A Convex Formulation for Learning Task Relationships in Multi-Task Learning
- Multi-Stage Multi-Task Feature Learning
- Constructing a Non-Negative Low Rank and Sparse Graph with Data-Adaptive Features
- Smoothing Proximal Gradient Method for General Structured Sparse Learning
- A Note on the SPICE Method
- On Better Exploring and Exploiting Task Relationships in Multi-Task Learning: Joint Model and Feature Learning
- Fast Overlapping Group Lasso
- On Multiplicative Multitask Feature Learning
- Structured LISTA for Multidimensional Harmonic Retrieval
- Robust Online Multi-Task Learning with Correlative and Personalized Structures
- A-SFS: Semi-supervised Feature Selection based on Multi-task Self-supervision
- Estimation of Convex Polytopes for Automatic Discovery of Charge State Transitions in Quantum Dot Arrays
- Shared Subspace Models for Multi-Group Covariance Estimation
- Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices
- Multivariate, Multistep Forecasting, Reconstruction and Feature Selection of Ocean Waves via Recurrent and Sequence-to-Sequence Networks
- Robust supervised classification and feature selection using a primal-dual method
- Deterministic tensor completion with hypergraph expanders
- Efficient Mixed-Norm Regularization: Algorithms and Safe Screening Methods
- Direct l_(2,p)-Norm Learning for Feature Selection
- A Multi-task Two-stream Spatiotemporal Convolutional Neural Network for Convective Storm Nowcasting
- Tackling Ordinal Regression Problem for Heterogeneous Data: Sparse and Deep Multi-Task Learning Approaches
- Transfer Learning in Information Criteria-based Feature Selection
- PCA-based Multi Task Learning: a Random Matrix Approach
- Framework for Multi-task Multiple Kernel Learning and Applications in Genome Analysis
- Spectral Algorithm for Low-rank Multitask Regression
- Learning Clinical Outcomes from Heterogeneous Genomic Data Sources
- Hyperspectral Imaging and Analysis for Sparse Reconstruction and Recognition
- Accounting for data heterogeneity in integrative analysis and prediction methods: An application to Chronic Obstructive Pulmonary Disease
- Causality and Robust Optimization
- Robust Dictionary based Data Representation
- Rank-Based Multi-task Learning for Fair Regression