Multi-Stage Multi-Task Feature Learning

Abstract

Multi-task sparse feature learning aims to improve the generalization performance by exploiting the shared features among tasks. It has been successfully applied to many applications including computer vision and biomedical informatics. Most of the existing multi-task sparse feature learning algorithms are formulated as a convex sparse regularization problem, which is usually suboptimal, due to its looseness for approximating an ${\ell }_0$-type regularizer. In this paper, we propose a non-convex formulation for multi-task sparse feature learning based on a novel non-convex regularizer. To solve the non-convex optimization problem, we propose a Multi-Stage Multi-Task Feature Learning (MSMTFL) algorithm; we also provide intuitive interpretations, detailed convergence and reproducibility analysis for the proposed algorithm. Moreover, we present a detailed theoretical analysis showing that MSMTFL achieves a better parameter estimation error bound than the convex formulation. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of MSMTFL in comparison with the state of the art multi-task sparse feature learning algorithms.

The short version appears in NIPS 2012

References in corpus (8)

• On the conditions used to prove oracle results for the Lasso
• Multi-Task Feature Learning Via Efficient l2,1-Norm Minimization
• Support union recovery in high-dimensional multivariate regression
• Strong oracle optimality of folded concave penalized estimation
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
• Taking Advantage of Sparsity in Multi-Task Learning
• Union Support Recovery in Multi-task Learning
• Multi-stage Convex Relaxation for Feature Selection

Cited by in corpus (9)

• A Survey on Multi-Task Learning
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
• On Better Exploring and Exploiting Task Relationships in Multi-Task Learning: Joint Model and Feature Learning
• On Multiplicative Multitask Feature Learning
• Robust Subspace Clustering via Tighter Rank Approximation
• Improving Spoken Language Understanding By Exploiting ASR N-best Hypotheses
• Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis
• Enhanced joint sparsity via Iterative Support Detection
• Multiple Influential Point Detection in High-Dimensional Spaces