Consistency of the group Lasso and multiple kernel learning

Abstract

We consider the least-square regression problem with regularization by a block 1-norm, i.e., a sum of Euclidean norms over spaces of dimensions larger than one. This problem, referred to as the group Lasso, extends the usual regularization by the 1-norm where all spaces have dimension one, where it is commonly referred to as the Lasso. In this paper, we study the asymptotic model consistency of the group Lasso. We derive necessary and sufficient conditions for the consistency of group Lasso under practical assumptions, such as model misspecification. When the linear predictors and Euclidean norms are replaced by functions and reproducing kernel Hilbert norms, the problem is usually referred to as multiple kernel learning and is commonly used for learning from heterogeneous data sources and for non linear variable selection. Using tools from functional analysis, and in particular covariance operators, we extend the consistency results to this infinite dimensional case and also propose an adaptive scheme to obtain a consistent model estimate, even when the necessary condition required for the non adaptive scheme is not satisfied.

References in corpus (11)

• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al
• Consistency of trace norm minimization

Cited by in corpus (133)

• Compressed Sensing of Block-Sparse Signals: Uncertainty Relations and Efficient Recovery
• Structured Compressed Sensing: From Theory to Applications
• Detecting causal associations in large nonlinear time series datasets
• Sparse Signal Recovery with Temporally Correlated Source Vectors Using Sparse Bayesian Learning
• Variable selection in nonparametric additive models
• Group Sparse Regularization for Deep Neural Networks
• Structured Variable Selection with Sparsity-Inducing Norms
• Support union recovery in high-dimensional multivariate regression
• Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations
• Breaking the Curse of Dimensionality with Convex Neural Networks
• A Selective Review of Group Selection in High-Dimensional Models
• Structured Sparse Principal Component Analysis
• Learning with Structured Sparsity
• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Sparsity in multiple kernel learning
• Consistency of trace norm minimization
• Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
• Sparse Attack Construction and State Estimation in the Smart Grid: Centralized and Distributed Models
• Regularization Techniques for Learning with Matrices
• Group Lasso with Overlaps: the Latent Group Lasso approach
• Taking Advantage of Sparsity in Multi-Task Learning
• Learning Multi-modal Similarity
• Regularized Least-Mean-Square Algorithms
• Complex Support Vector Machines for Regression and Quaternary Classification
• Near-Oracle Performance of Greedy Block-Sparse Estimation Techniques from Noisy Measurements
• Forward - Backward Greedy Algorithms for Atomic Norm Regularization
• Fast learning rate of multiple kernel learning: Trade-off between sparsity and smoothness
• Nonparametric sparsity and regularization
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Trace Lasso: a trace norm regularization for correlated designs
• High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning
• Network classification with applications to brain connectomics
• WebCaricature: a benchmark for caricature recognition
• Network Granger Causality with Inherent Grouping Structure
• Group Lasso for high dimensional sparse quantile regression models
• Learning Model-Based Sparsity via Projected Gradient Descent
• Learning the Conditional Independence Structure of Stationary Time Series: A Multitask Learning Approach
• Recursive ${\ell }_{1,\infty }$ Group lasso
• Sparse Group Restricted Boltzmann Machines
• Sparsity with sign-coherent groups of variables via the cooperative-Lasso
• A Binary Classification Framework for Two-Stage Multiple Kernel Learning
• Forest Sparsity for Multi-channel Compressive Sensing
• Adaptive Lasso and group-Lasso for functional Poisson regression
• Convolutional Matching Pursuit and Dictionary Training
• Stacked Penalized Logistic Regression for Selecting Views in Multi-View Learning
• Sparsity-accuracy trade-off in MKL
• The Benefit of Group Sparsity
• Multiclass Sparse Discriminant Analysis
• Compressive MUSIC: A Missing Link Between Compressive Sensing and Array Signal Processing
• Tight Measurement Bounds for Exact Recovery of Structured Sparse Signals
• Regularization with the Smooth-Lasso procedure
• Model-Consistent Sparse Estimation through the Bootstrap
• Group Sparse Additive Models
• Sparse and Smooth Signal Estimation: Convexification of L0 Formulations
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Group sparse optimization via ${\ell }_{p,q}$ regularization
• Sparse quadratic classification rules via linear dimension reduction
• Signal Recovery in Unions of Subspaces with Applications to Compressive Imaging
• Multiple Kernel Learning from Noisy Labels by Stochastic Programming
• Learning Heteroscedastic Models by Convex Programming under Group Sparsity
• Joint Base Station Clustering and Beamformer Design for Partial Coordinated Transmission in Heterogenous Networks
• A General Framework of Dual Certificate Analysis for Structured Sparse Recovery Problems
• Accuracy guaranties for ${\ell }_1$ recovery of block-sparse signals
• Asymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP)
• PAC-Bayesian aggregation and multi-armed bandits
• Dimension-free tail inequalities for sums of random matrices
• Positive Definite Kernels in Machine Learning
• A Complete Analysis of the l_1,p Group-Lasso
• Group Lasso estimation of high-dimensional covariance matrices
• Kernel machines with two layers and multiple kernel learning
• Fast non-coplanar beam orientation optimization based on group sparsity
• Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
• Motion-robust free-running volumetric cardiovascular MRI
• Exact recovery of the support of piecewise constant images via total variation regularization
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• Maximin Analysis of Message Passing Algorithms for Recovering Block Sparse Signals
• Multivariate functional group sparse regression: functional predictor selection
• Online Learning for Matrix Factorization and Sparse Coding
• Efficient Mixed-Norm Regularization: Algorithms and Safe Screening Methods
• Two-step estimation of high dimensional additive models
• ShareBoost: Efficient Multiclass Learning with Feature Sharing
• Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition
• Learning Schizophrenia Imaging Genetics Data Via Multiple Kernel Canonical Correlation Analysis
• Improved Support Recovery Guarantees for the Group Lasso With Applications to Structural Health Monitoring
• Oracle Inequalities and Optimal Inference under Group Sparsity
• The Degrees of Freedom of Partly Smooth Regularizers
• Convex vs nonconvex approaches for sparse estimation: GLasso, Multiple Kernel Learning and Hyperparameter GLasso
• Non-Sparse Regularization for Multiple Kernel Learning
• A Max-Product EM Algorithm for Reconstructing Markov-tree Sparse Signals from Compressive Samples
• The Smooth-Lasso and other ${\ell }_1+{\ell }_2$-penalized methods
• The high order block RIP condition for signal recovery
• Robust Kernel (Cross-) Covariance Operators in Reproducing Kernel Hilbert Space toward Kernel Methods
• Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models
• JOBS: Joint-Sparse Optimization from Bootstrap Samples
• The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso
• GEP-MSCRA for computing the group zero-norm regularized least squares estimator
• Multi-task Learning of Order-Consistent Causal Graphs
• Learning Random Kernel Approximations for Object Recognition
• On the Sample Complexity of Graphical Model Selection for Non-Stationary Processes
• Grouped Variable Selection via Nested Spike and Slab Priors
• Efficient Multi-Template Learning for Structured Prediction
• Jointly Sparse Global SIMPLS Regression
• Sharp Support Recovery from Noisy Random Measurements by L1 minimization
• The Degrees of Freedom of the Group Lasso
• A Communication-Efficient Parallel Method for Group-Lasso
• Sparse Feature Selection in Kernel Discriminant Analysis via Optimal Scoring
• Dynamic Networks with Multi-scale Temporal Structure
• Transfer Learning Using Feature Selection
• Sparse Multiple Kernel Learning: Support Identification via Mirror Stratifiability
• Equivalent Lipschitz surrogates for zero-norm and rank optimization problems
• Flexible Variable Selection for Recovering Sparsity in Nonadditive Nonparametric Models
• A Multiple Hypothesis Testing Approach to Low-Complexity Subspace Unmixing
• Resolving Out-of-Vocabulary Words with Bilingual Embeddings in Machine Translation
• Learning a Common Substructure of Multiple Graphical Gaussian Models
• Smooth Bilevel Programming for Sparse Regularization
• A short note on extension theorems and their connection to universal consistency in machine learning
• Sharp Oracle Inequalities for Low-complexity Priors
• Multiple Kernel Learning and Automatic Subspace Relevance Determination for High-dimensional Neuroimaging Data
• Semi-supervised Ranking Pursuit
• Bridging Information Criteria and Parameter Shrinkage for Model Selection
• Sharp Threshold for Multivariate Multi-Response Linear Regression via Block Regularized Lasso
• Multiple Kernel Learning in the Primal for Multi-modal Alzheimer's Disease Classification
• Sharp Convergence Rate and Support Consistency of Multiple Kernel Learning with Sparse and Dense Regularization
• Sparse high-dimensional varying coefficient model: non-asymptotic minimax study
• On the Linear Convergence of the Approximate Proximal Splitting Method for Non-Smooth Convex Optimization
• High-dimensional regression with unknown variance
• Fast projections onto mixed-norm balls with applications
• Learning rates for the risk of kernel based quantile regression estimators in additive models
• Efficient algorithms for robust recovery of images from compressed data
• Fast Identification of Biological Pathways Associated with a Quantitative Trait Using Group Lasso with Overlaps
• Covariate-Adjusted Tensor Classification in High-Dimensions
• A New Greedy Algorithm for Multiple Sparse Regression
• Proximal methods for the latent group lasso penalty