Ultrafast single-channel machine vision based on neuro-inspired photonic computing

Abstract

High-speed machine vision is increasing its importance in both scientific and technological applications. Neuro-inspired photonic computing is a promising approach to speed-up machine vision processing with ultralow latency. However, the processing rate is fundamentally limited by the low frame rate of image sensors, typically operating at tens of hertz. Here, we propose an image-sensor-free machine vision framework, which optically processes real-world visual information with only a single input channel, based on a random temporal encoding technique. This approach allows for compressive acquisitions of visual information with a single channel at gigahertz rates, outperforming conventional approaches, and enables its direct photonic processing using a photonic reservoir computer in a time domain. We experimentally demonstrate that the proposed approach is capable of high-speed image recognition and anomaly detection, and furthermore, it can be used for high-speed imaging. The proposed approach is multipurpose and can be extended for a wide range of applications, including tracking, controlling, and capturing sub-nanosecond phenomena.

arXiv comment30 pages, 12 figures

References in corpus (24)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Machine learning and the physical sciences
• Recent Advances in Physical Reservoir Computing: A Review
• Parallel convolution processing using an integrated photonic tensor core
• Photonics for artificial intelligence and neuromorphic computing
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• Neuromorphic Silicon Photonic Networks
• Ghost imaging with a single detector
• Optoelectronic Reservoir Computing
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Single-chip photonic deep neural network for instantaneous image classification
• Hadamard single-pixel imaging versus Fourier single-pixel imaging
• Wave Physics as an Analog Recurrent Neural Network
• An optical neural network using less than 1 photon per multiplication
• Image sensing with multilayer, nonlinear optical neural networks
• Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
• Delocalized Photonic Deep Learning on the Internet's Edge
• Human action recognition with a large-scale brain-inspired photonic computer
• Physical deep learning based on optimal control of dynamical systems
• Gigahertz-rate random speckle projection for high-speed single-pixel image classification
• Anomaly Detection in Images
• Short-wavelength Reverberant Wave Systems for Physical Realization of Reservoir Computing

Cited by in corpus (2)

• Neural Network Methods for Radiation Detectors and Imaging
• Robust Quantum Reservoir Computing for Molecular Property Prediction