Physical deep learning based on optimal control of dynamical systems

Abstract

Deep learning is the backbone of artificial intelligence technologies, and it can be regarded as a kind of multilayer feedforward neural network. An essence of deep learning is information propagation through layers. This suggests that there is a connection between deep neural networks and dynamical systems in the sense that information propagation is explicitly modeled by the time-evolution of dynamical systems. In this study, we perform pattern recognition based on the optimal control of continuous-time dynamical systems, which is suitable for physical hardware implementation. The learning is based on the adjoint method to optimally control dynamical systems, and the deep (virtual) network structures based on the time evolution of the systems are used for processing input information. As a key example, we apply the dynamics-based recognition approach to an optoelectronic delay system and demonstrate that the use of the delay system allows for image recognition and nonlinear classifications using only a few control signals. This is in contrast to conventional multilayer neural networks, which require a large number of weight parameters to be trained. The proposed approach provides insight into the mechanisms of deep network processing in the framework of an optimal control problem and presents a pathway for realizing physical computing hardware.

11 pages, 9 figures

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• An overview of gradient descent optimization algorithms
• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Recent Advances in Physical Reservoir Computing: A Review
• Neuromorphic Silicon Photonic Networks
• On the Number of Linear Regions of Deep Neural Networks
• Stable Architectures for Deep Neural Networks
• Exponential expressivity in deep neural networks through transient chaos
• Neural Network with Unbounded Activation Functions is Universal Approximator
• Complex Dynamics and Synchronization of Delayed-Feedback Nonlinear Oscillators
• Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
• Transfer learning for nonlinear dynamics and its application to fluid turbulence

Cited by in corpus (6)

• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Roadmap for Unconventional Computing with Nanotechnology
• Ultrafast single-channel machine vision based on neuro-inspired photonic computing
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Blending Optimal Control and Biologically Plausible Learning for Noise-Robust Physical Neural Networks
• Photonic neural field on a silicon chip: large-scale, high-speed neuro-inspired computing and sensing