McKernel: A Library for Approximate Kernel Expansions in Log-linear Time
arXiv:1702.08159
Abstract
McKernel introduces a framework to use kernel approximates in the mini-batch setting with Stochastic Gradient Descent (SGD) as an alternative to Deep Learning. Based on Random Kitchen Sinks [Rahimi and Recht 2007], we provide a C++ library for Large-scale Machine Learning. It contains a CPU optimized implementation of the algorithm in [Le et al. 2013], that allows the computation of approximated kernel expansions in log-linear time. The algorithm requires to compute the product of matrices Walsh Hadamard. A cache friendly Fast Walsh Hadamard that achieves compelling speed and outperforms current state-of-the-art methods has been developed. McKernel establishes the foundation of a new architecture of learning that allows to obtain large-scale non-linear classification combining lightning kernel expansions and a linear classifier. It travails in the mini-batch setting working analogously to Neural Networks. We show the validity of our method through extensive experiments on MNIST and FASHION MNIST [Xiao et al. 2017].
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- AutoAugment: Learning Augmentation Policies from Data
- Practical and Optimal LSH for Angular Distance
- Generalization Properties of Learning with Random Features
- A Bayesian Data Augmentation Approach for Learning Deep Models
- Stochastic Variational Deep Kernel Learning
- Fast Two-Sample Testing with Analytic Representations of Probability Measures
- Theory II: Landscape of the Empirical Risk in Deep Learning
- Adding One Neuron Can Eliminate All Bad Local Minima
- A la Carte - Learning Fast Kernels
- Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex