On Fast Leverage Score Sampling and Optimal Learning
arXiv:1810.13258
Abstract
Leverage score sampling provides an appealing way to perform approximate computations for large matrices. Indeed, it allows to derive faithful approximations with a complexity adapted to the problem at hand. Yet, performing leverage scores sampling is a challenge in its own right requiring further approximations. In this paper, we study the problem of leverage score sampling for positive definite matrices defined by a kernel. Our contribution is twofold. First we provide a novel algorithm for leverage score sampling and second, we exploit the proposed method in statistical learning by deriving a novel solver for kernel ridge regression. Our main technical contribution is showing that the proposed algorithms are currently the most efficient and accurate for these problems.
References in corpus (11)
- Searching for Exotic Particles in High-Energy Physics with Deep Learning
- A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets
- Generalization Properties of Learning with Random Features
- Recursive Sampling for the Nyström Method
- Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
- Fast Randomized Kernel Methods With Statistical Guarantees
- A Consistent Regularization Approach for Structured Prediction
- Learning with SGD and Random Features
- Second-Order Kernel Online Convex Optimization with Adaptive Sketching
- Consistent Multitask Learning with Nonlinear Output Relations
- A Structured Prediction Approach for Label Ranking