Transferable polychromatic optical encoder for neural networks

Abstract

Artificial neural networks (ANNs) have fundamentally transformed the field of computer vision, providing unprecedented performance. However, these ANNs for image processing demand substantial computational resources, often hindering real-time operation. In this paper, we demonstrate an optical encoder that can perform convolution simultaneously in three color channels during the image capture, effectively implementing several initial convolutional layers of a ANN. Such an optical encoding results in ~24,000 times reduction in computational operations, with a state-of-the art classification accuracy (~73.2%) in free-space optical system. In addition, our analog optical encoder, trained for CIFAR-10 data, can be transferred to the ImageNet subset, High-10, without any modifications, and still exhibits moderate accuracy. Our results evidence the potential of hybrid optical/digital computer vision system in which the optical frontend can pre-process an ambient scene to reduce the energy and latency of the whole computer vision system.

21 pages, 4 figures, 2 tables

References in corpus (10)

• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• The physics of optical computing
• Image sensing with multilayer, nonlinear optical neural networks
• Deep Learning with Coherent VCSEL Neural Networks
• Nonlinear optical encoding enabled by recurrent linear scattering
• All-optical image classification through unknown random diffusers using a single-pixel diffractive network
• Time-lapse image classification using a diffractive neural network
• Using Scalable Computer Vision to Automate High-throughput Semiconductor Characterization
• Optical convolutional neural network with atomic nonlinearity
• Compressed Meta-Optical Encoder for Image Classification