Kernel computations from large-scale random features obtained by Optical Processing Units

Abstract

Approximating kernel functions with random features (RFs)has been a successful application of random projections for nonparametric estimation. However, performing random projections presents computational challenges for large-scale problems. Recently, a new optical hardware called Optical Processing Unit (OPU) has been developed for fast and energy-efficient computation of large-scale RFs in the analog domain. More specifically, the OPU performs the multiplication of input vectors by a large random matrix with complex-valued i.i.d. Gaussian entries, followed by the application of an element-wise squared absolute value operation - this last nonlinearity being intrinsic to the sensing process. In this paper, we show that this operation results in a dot-product kernel that has connections to the polynomial kernel, and we extend this computation to arbitrary powers of the feature map. Experiments demonstrate that the OPU kernel and its RF approximation achieve competitive performance in applications using kernel ridge regression and transfer learning for image classification. Crucially, thanks to the use of the OPU, these results are obtained with time and energy savings.

5 pages, 3 figures, submitted to ICASSP 2020

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium
• Optical Reservoir Computing using multiple light scattering for chaotic systems prediction
• Random Projections through multiple optical scattering: Approximating kernels at the speed of light
• Random Feature Expansions for Deep Gaussian Processes
• Scaling up Echo-State Networks with multiple light scattering

Cited by in corpus (8)

• Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
• Generalisation error in learning with random features and the hidden manifold model
• Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
• Reservoir Computing meets Recurrent Kernels and Structured Transforms
• Photonic kernel machine learning for ultrafast spectral analysis
• Machine learning via relativity-inspired quantum dynamics
• Photonic Differential Privacy with Direct Feedback Alignment
• Improvement of image classification by multiple optical scattering