Sparsity in multiple kernel learning

Abstract

The problem of multiple kernel learning based on penalized empirical risk minimization is discussed. The complexity penalty is determined jointly by the empirical $L_2$ norms and the reproducing kernel Hilbert space (RKHS) norms induced by the kernels with a data-driven choice of regularization parameters. The main focus is on the case when the total number of kernels is large, but only a relatively small number of them is needed to represent the target function, so that the problem is sparse. The goal is to establish oracle inequalities for the excess risk of the resulting prediction rule showing that the method is adaptive both to the unknown design distribution and to the sparsity of the problem.

Published in at http://dx.doi.org/10.1214/10-AOS825 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (7)

• Simultaneous analysis of Lasso and Dantzig selector
• Consistency of the group Lasso and multiple kernel learning
• Component selection and smoothing in multivariate nonparametric regression
• High-dimensional additive modeling
• Statistical performance of support vector machines
• The Dantzig selector and sparsity oracle inequalities
• Sparse recovery in convex hulls via entropy penalization

Cited by in corpus (53)

• A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
• Fast learning rate of multiple kernel learning: Trade-off between sparsity and smoothness
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Nonparametric sparsity and regularization
• Minimax-optimal nonparametric regression in high dimensions
• Group Lasso for high dimensional sparse quantile regression models
• Tight conditions for consistency of variable selection in the context of high dimensionality
• The Local Rademacher Complexity of Lp-Norm Multiple Kernel Learning
• Electricity Market Forecasting via Low-Rank Multi-Kernel Learning
• PAC-Bayesian Estimation and Prediction in Sparse Additive Models
• Doubly Decomposing Nonparametric Tensor Regression
• Improved Bound for the Nystrom's Method and its Application to Kernel Classification
• Minimax-optimal rates for sparse additive models over kernel classes via convex programming
• Nonparametric Basis Pursuit via Sparse Kernel-based Learning
• A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound
• Learning Heteroscedastic Models by Convex Programming under Group Sparsity
• Sparse Additive Functional and Kernel CCA
• Scalable Matrix-valued Kernel Learning for High-dimensional Nonlinear Multivariate Regression and Granger Causality
• The Group Square-Root Lasso: Theoretical Properties and Fast Algorithms
• Nonparametric Regression with Adaptive Truncation via a Convex Hierarchical Penalty
• Two-step estimation of high dimensional additive models
• Minimax Optimal Rates of Estimation in High Dimensional Additive Models: Universal Phase Transition
• Functional Linear Regression with Mixed Predictors
• Regularizers for Structured Sparsity
• Structured nonlinear variable selection
• Learning Sparse Additive Models with Interactions in High Dimensions
• Regularizing Double Machine Learning in Partially Linear Endogenous Models
• Estimation and Uniform Inference in Sparse High-Dimensional Additive Models
• Variable selection in high-dimensional additive models based on norms of projections
• Faithful Variable Screening for High-Dimensional Convex Regression
• Functional Autoregressive Processes in Reproducing Kernel Hilbert Spaces
• Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods
• Post-Regularization Confidence Bands for Ordinary Differential Equations
• Fast Convergence Rate of Multiple Kernel Learning with Elastic-net Regularization
• A theory of nonparametric regression in the presence of complex nuisance components
• Estimates on Learning Rates for Multi-Penalty Distribution Regression
• Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
• Detecting Nonlinear Causality in Multivariate Time Series with Sparse Additive Models
• Extreme Eigenvalues of Nonlinear Correlation Matrices with Applications to Additive Models
• Minimax Optimal Estimation in Partially Linear Additive Models under High Dimension
• Learning rates for partially linear support vector machine in high dimensions
• Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error
• Sparse additive regression on a regular lattice
• Kernel Ordinary Differential Equations
• Post-selection inference on high-dimensional varying-coefficient quantile regression model
• Sparse Multiple Kernel Learning with Geometric Convergence Rate
• Fast Learning Rate of lp-MKL and its Minimax Optimality
• How many moments does MMD compare?
• Learning rates for classification with Gaussian kernels
• Adaptive Estimation In High-Dimensional Additive Models With Multi-Resolution Group Lasso
• A Theoretical Framework for Bayesian Nonparametric Regression
• A short note on extension theorems and their connection to universal consistency in machine learning
• Learning with Optimal Interpolation Norms