A Survey of Neuromorphic Computing and Neural Networks in Hardware

Abstract

Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture. This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems. The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities. In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history. We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications. We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled. The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed.

References in corpus (21)

• Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• Integration of nanoscale memristor synapses in neuromorphic computing architectures
• Filamentary Switching: Synaptic Plasticity through Device Volatility
• Synchronous Behavior of Two Coupled Electronic Neurons
• A CMOS Spiking Neuron for Brain-Inspired Neural Networks with Resistive Synapses and In-Situ Learning
• Solitary and Coupled Semiconductor Ring Lasers as Optical Spiking Neurons
• Probabilistic Deep Spiking Neural Systems Enabled by Magnetic Tunnel Junction
• Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition
• Spin Neurons: A Possible Path to Energy-Efficient Neuromorphic Computers
• Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms
• A Neuromorphic VLSI Design for Spike Timing and Rate Based Synaptic Plasticity
• Neuro-Fuzzy Computing System with the Capacity of Implementation on Memristor-Crossbar and Optimization-Free Hardware Training
• Ultra Low Energy Analog Image Processing Using Spin Neurons
• A New Learning Method for Inference Accuracy, Core Occupation, and Performance Co-optimization on TrueNorth Chip
• Spin switches for compact implementation of neuron and synapse
• Implementing robust neuromodulation in neuromorphic circuits
• Event management for large scale event-driven digital hardware spiking neural networks
• Gibbs Sampling with Low-Power Spiking Digital Neurons
• Memristor-based Synaptic Networks and Logical Operations Using In-Situ Computing
• A Biological-Realtime Neuromorphic System in 28 nm CMOS using Low-Leakage Switched Capacitor Circuits

Cited by in corpus (65)

• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
• Neuro-memristive Circuits for Edge Computing: A review
• An Artificial Neuron Implemented on an Actual Quantum Processor
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Prospects and applications of photonic neural networks
• Intelligent metaphotonics empowered by machine learning
• Survey of Machine Learning Accelerators
• A new Taxonomy of Continuous Global Optimization Algorithms
• Machine Learning for Security in Vehicular Networks: A Comprehensive Survey
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment
• Unsupervised Heart-rate Estimation in Wearables With Liquid States and A Probabilistic Readout
• Autonomous Probabilistic Coprocessing with Petaflips per Second
• Neuromorphic Hardware learns to learn
• Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligence
• Neuromorphic scaling advantages for energy-efficient random walk computation
• Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques
• RANC: Reconfigurable Architecture for Neuromorphic Computing
• NeuroHammer: Inducing Bit-Flips in Memristive Crossbar Memories
• Monadic Pavlovian associative learning in a backpropagation-free photonic network
• Optoelectronic Intelligence
• Evaluating Ising Processing Units with Integer Programming
• Constant-Depth and Subcubic-Size Threshold Circuits for Matrix Multiplication
• Stochasticity and Robustness in Spiking Neural Networks
• Neuromorphic Computing for Content-based Image Retrieval
• Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients
• Rethinking Non-idealities in Memristive Crossbars for Adversarial Robustness in Neural Networks
• Entangled Quantum Memristors
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• Synaptic Delays for Temporal Feature Detection in Dynamic Neuromorphic Processors
• An active dendritic tree can mitigate fan-in limitations in superconducting neurons
• Artificial Spin Ice: A Tutorial on Design and Control of Geometry, Microstate, Magnon Dynamics & Neuromorphic Computing
• Efficiency-driven Hardware Optimization for Adversarially Robust Neural Networks
• Many-Joint Robot Arm Control with Recurrent Spiking Neural Networks
• Improving STDP-based Visual Feature Learning with Whitening
• Spike-based primitives for graph algorithms
• Perspectives of Using Oscillators for Computing and Signal Processing
• Brain-Inspired Hardware for Artificial Intelligence: Accelerated Learning in a Physical-Model Spiking Neural Network
• Superconducting Optoelectronic Neurons II: Receiver Circuits
• Quantum Technologies: A Review of the Patent Landscape
• Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization
• Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects
• Combining Spiking Neural Network and Artificial Neural Network for Enhanced Image Classification
• Design and Simulation of Memristor-Based Artificial Neural Network for Bidirectional Adaptive Neural Interface
• Neuromorphic Processing and Sensing: Evolutionary Progression of AI to Spiking
• Spintronics for neuromorphic computing
• PeleNet: A Reservoir Computing Framework for Loihi
• A Multilayer Neural Network Merging Image Preprocessing and Pattern Recognition by Integrating Diffusion and Drift Memristors
• Energy-Efficient CMOS Memristive Synapses for Mixed-Signal Neuromorphic System-on-a-Chip
• Benchmarking Physical Performance of Neural Inference Circuits
• Rebooting Neuromorphic Hardware Design -- A Complexity Engineering Approach
• The largest cognitive systems will be optoelectronic
• MERIT: Tensor Transform for Memory-Efficient Vision Processing on Parallel Architectures
• Spiking Linear Dynamical Systems on Neuromorphic Hardware for Low-Power Brain-Machine Interfaces
• Confusion-based rank similarity filters for computationally-efficient machine learning on high dimensional data
• Non-binary artificial neuron with phase variation implemented on a quantum computer
• Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations
• Building Reservoir Computing Hardware Using Low Energy-Barrier Magnetics
• A Non-equilibrium Thermodynamic Framework of Consciousness
• A Neural Architecture Search based Framework for Liquid State Machine Design
• Cerebral cortical communication overshadows computational energy-use, but these combine to predict synapse number
• A Memristor-Based Optimization Framework for AI Applications
• Understanding Attention: In Minds and Machines
• Training Dynamic Exponential Family Models with Causal and Lateral Dependencies for Generalized Neuromorphic Computing