Self-training superconducting neuromorphic circuits using reinforcement learning rules

Abstract

Reinforcement learning algorithms are used in a wide range of applications, from gaming and robotics to autonomous vehicles. In this paper we describe a set of reinforcement learning-based local weight update rules and their implementation in superconducting hardware. Using SPICE circuit simulations, we implement a small-scale neural network with a learning time of order one nanosecond. This network can be trained to learn new functions simply by changing the target output for a given set of inputs, without the need for any external adjustments to the network. In this implementation the weights are adjusted based on the current state of the overall network response and locally stored information about the previous action. This removes the need to program explicit weight values in these networks, which is one of the primary challenges that analog hardware implementations of neural networks face. The adjustment of weights is based on a global reinforcement signal that obviates the need for circuitry to back-propagate errors.

15 pages, 6 figures

References in corpus (11)

• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Josephson junction simulation of neurons
• Synchronization dynamics on the picosecond timescale in coupled Josephson junction neurons
• Microwave Superconductivity
• Fan-out and Fan-in properties of superconducting neuromorphic circuits
• Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation
• Propagation of Picosecond Pulses on Superconducting Transmission Line Interconnects
• Unsupervised SFQ-Based Spiking Neural Network
• Deep Neuromorphic Networks with Superconducting Single Flux Quanta
• Hybrid Synaptic Structure for Spiking Neural Network Realization
• Roadmap to Neuromorphic Computing with Emerging Technologies