Trainable and Dynamic Computing: Error Backpropagation through Physical Media

Abstract

Machine learning algorithms, and more in particular neural networks, arguably experience a revolution in terms of performance. Currently, the best systems we have for speech recognition, computer vision and similar problems are based on neural networks, trained using the half-century old backpropagation algorithm. Despite the fact that neural networks are a form of analog computers, they are still implemented digitally for reasons of convenience and availability. In this paper we demonstrate how we can design physical linear dynamic systems with non-linear feedback as a generic platform for dynamic, neuro-inspired analog computing. We show that a crucial advantage of this setup is that the error backpropagation can be performed physically as well, which greatly speeds up the optimisation process. As we show in this paper, using one experimentally validated and one conceptual example, such systems may be the key to providing a relatively straightforward mechanism for constructing highly scalable, fully dynamic analog computers.

References in corpus (1)

• Random feedback weights support learning in deep neural networks

Cited by in corpus (14)

• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Training deep neural networks for the inverse design of nanophotonic structures
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Training of photonic neural networks through in situ backpropagation
• Wave Physics as an Analog Recurrent Neural Network
• Embodiment of Learning in Electro-Optical Signal Processors
• Self-learning Machines based on Hamiltonian Echo Backpropagation
• Deep Photonic Reservoir Computer for Speech Recognition
• Training and pattern recognition by an opto-magnetic neural network
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Towards Trainable Media: Using Waves for Neural Network-Style Training
• A universal approximation theorem for nonlinear resistive networks
• Towards Deep Physical Reservoir Computing Through Automatic Task Decomposition And Mapping
• Online training and pruning of multi-wavelength photonic neural networks