Long short-term memory networks in memristor crossbars

Abstract

Recent breakthroughs in recurrent deep neural networks with long short-term memory (LSTM) units has led to major advances in artificial intelligence. State-of-the-art LSTM models with significantly increased complexity and a large number of parameters, however, have a bottleneck in computing power resulting from limited memory capacity and data communication bandwidth. Here we demonstrate experimentally that LSTM can be implemented with a memristor crossbar, which has a small circuit footprint to store a large number of parameters and in-memory computing capability that circumvents the 'von Neumann bottleneck'. We illustrate the capability of our system by solving real-world problems in regression and classification, which shows that memristor LSTM is a promising low-power and low-latency hardware platform for edge inference.

References in corpus (5)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
• Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices
• Mixed-Precision In-Memory Computing
• The Microsoft 2017 Conversational Speech Recognition System

Cited by in corpus (15)

• Photonics for artificial intelligence and neuromorphic computing
• Mixed-precision deep learning based on computational memory
• Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks
• A memristive deep belief neural network based on silicon synapses
• Picosecond Multilevel Resistive Switching in Tantalum Oxide Thin Films
• Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array
• Dynamic-quenching of a single-photon avalanche photodetector using an adaptive resistive switch
• Efficient Training of the Memristive Deep Belief Net Immune to Non-Idealities of the Synaptic Devices
• Design and Characterization of Superconducting Nanowire-Based Processors for Acceleration of Deep Neural Network Training
• Breaking the quantum PIN code of atomic synapses
• Artificial Nanophotonic Neuron with Internal Memory for Biologically Inspired and Reservoir Network Computing
• Learning to Approximate Functions Using Nb-doped SrTiO${}_3$ Memristors
• Quantum transport properties of tantalum-oxide resistive switching filaments
• Learning Chaotic Dynamics with Neuromorphic Network Dynamics
• 2022 Roadmap on Neuromorphic Computing and Engineering