Random Projections through multiple optical scattering: Approximating kernels at the speed of light

Abstract

Random projections have proven extremely useful in many signal processing and machine learning applications. However, they often require either to store a very large random matrix, or to use a different, structured matrix to reduce the computational and memory costs. Here, we overcome this difficulty by proposing an analog, optical device, that performs the random projections literally at the speed of light without having to store any matrix in memory. This is achieved using the physical properties of multiple coherent scattering of coherent light in random media. We use this device on a simple task of classification with a kernel machine, and we show that, on the MNIST database, the experimental results closely match the theoretical performance of the corresponding kernel. This framework can help make kernel methods practical for applications that have large training sets and/or require real-time prediction. We discuss possible extensions of the method in terms of a class of kernels, speed, memory consumption and different problems.

References in corpus (1)

• Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques

Cited by in corpus (38)

• The physics of optical computing
• Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
• Scalable Optical Learning Operator
• Heuristic Recurrent Algorithms for Photonic Ising Machines
• Optical Reservoir Computing using multiple light scattering for chaotic systems prediction
• Nonlinear optical encoding enabled by recurrent linear scattering
• Generalisation error in learning with random features and the hidden manifold model
• Noise-enhanced spatial-photonic Ising machine
• Readout of fluorescence functional signals through highly scattering tissue
• Photonic convolutional neural networks using integrated diffractive optics
• Photonic Extreme Learning Machine based on frequency multiplexing
• Deep Hybrid Scattering Image Learning
• An array of microresonators as a Photonic Extreme Learning Machine
• Wave-based extreme deep learning based on non-linear time-Floquet entanglement
• Scaling up Echo-State Networks with multiple light scattering
• Quantized Compressive K-Means
• Kernel computations from large-scale random features obtained by Optical Processing Units
• Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
• ACDC: A Structured Efficient Linear Layer
• Deep Photonic Reservoir Computer for Speech Recognition
• Reservoir Computing meets Recurrent Kernels and Structured Transforms
• Photonic kernel machine learning for ultrafast spectral analysis
• AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
• Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment
• Don't take it lightly: Phasing optical random projections with unknown operators
• High-Fidelity and High-Speed Wavefront Shaping by Leveraging Complex Media
• Total Least Squares Phase Retrieval
• Light-in-the-loop: using a photonics co-processor for scalable training of neural networks
• Integrated Random Projection and Dimensionality Reduction by Propagating Light in Photonic Lattices
• Ultrafast photonic reinforcement learning based on laser chaos
• Machine learning via relativity-inspired quantum dynamics
• Statistical optimality conditions for compressive ensembles
• XConv: Low-memory stochastic backpropagation for convolutional layers
• Gaussian Compression Stream: Principle and Preliminary Results
• Improvement of image classification by multiple optical scattering
• Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment
• Optical Proof of Work
• What Happens on the Edge, Stays on the Edge: Toward Compressive Deep Learning