Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence
arXiv:2106.01288 · doi:10.1109/JPROC.2023.3273520
Abstract
While Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance. One of these avenues is the exploration of alternative brain-inspired computing architectures that aim at achieving the flexibility and computational efficiency of biological neural processing systems. Within this context, neuromorphic engineering represents a paradigm shift in computing based on the implementation of spiking neural network architectures in which processing and memory are tightly co-located. In this paper, we provide a comprehensive overview of the field, highlighting the different levels of granularity at which this paradigm shift is realized and comparing design approaches that focus on replicating natural intelligence (bottom-up) versus those that aim at solving practical artificial intelligence applications (top-down). First, we present the analog, mixed-signal and digital circuit design styles, identifying the boundary between processing and memory through time multiplexing, in-memory computation, and novel devices. Then, we highlight the key tradeoffs for each of the bottom-up and top-down design approaches, survey their silicon implementations, and carry out detailed comparative analyses to extract design guidelines. Finally, we identify necessary synergies and missing elements required to achieve a competitive advantage for neuromorphic systems over conventional machine-learning accelerators in edge computing applications, and outline the key ingredients for a framework toward neuromorphic intelligence.
arXiv commentAccepted for publication in Proceedings of the IEEE
References in corpus (56)
- Deep Learning in Neural Networks: An Overview
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Theoretical Models of Learning to Learn
- Memory and information processing in neuromorphic systems
- DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
- A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)
- Brain-inspired computing: We need a master plan
- Integration of nanoscale memristor synapses in neuromorphic computing architectures
- SuperSpike: Supervised learning in multi-layer spiking neural networks
- SLAYER: Spike Layer Error Reassignment in Time
- A 0.086-mm 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS
- The Computational Limits of Deep Learning
- Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules
- Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
- The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
- Towards Biologically Plausible Deep Learning
- Spiking Neural Networks Hardware Implementations and Challenges: a Survey
- Neuromorphic Deep Learning Machines
- A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields
- Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
- Benchmarking TinyML Systems: Challenges and Direction
- S4NN: temporal backpropagation for spiking neural networks with one spike per neuron
- Direct Feedback Alignment Provides Learning in Deep Neural Networks
- MorphIC: A 65-nm 738k-Synapse/mm Quad-Core Binary-Weight Digital Neuromorphic Processor with Stochastic Spike-Driven Online Learning
- Adaptive Extreme Edge Computing for Wearable Devices
- ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales
- Biologically Inspired Spiking Neurons : Piecewise Linear Models and Digital Implementation
- An electronic neuromorphic system for real-time detection of High Frequency Oscillations (HFOs) in intracranial EEG
- Training Neural Networks with Local Error Signals
- Real-time ultra-low power ECG anomaly detection using an event-driven neuromorphic processor
- An Accelerated LIF Neuronal Network Array for a Large Scale Mixed-Signal Neuromorphic Architecture
- A recipe for creating ideal hybrid memristive-CMOS neuromorphic computing systems
- Ultra-Low-Power FDSOI Neural Circuits for Extreme-Edge Neuromorphic Intelligence
- A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation
- SpiNNaker 2: A 10 Million Core Processor System for Brain Simulation and Machine Learning
- Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks
- The importance of space and time in neuromorphic cognitive agents
- A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores
- Low-Power Low-Latency Keyword Spotting and Adaptive Control with a SpiNNaker 2 Prototype and Comparison with Loihi
- Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
- Closed-loop spiking control on a neuromorphic processor implemented on the iCub
- RANC: Reconfigurable Architecture for Neuromorphic Computing
- Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
- A Theoretical Framework for Target Propagation
- An Ultralow Leakage Synaptic Scaling Homeostatic Plasticity Circuit With Configurable Time Scales up to 100 ks
- Understanding Synthetic Gradients and Decoupled Neural Interfaces
- Principled Training of Neural Networks with Direct Feedback Alignment
- EDEN: A high-performance, general-purpose, NeuroML-based neural simulator
- Equilibrium Propagation with Continual Weight Updates
- NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems
- Disentanglement with Biological Constraints: A Theory of Functional Cell Types
- A New Look at Spike-Timing-Dependent Plasticity Networks for Spatio-Temporal Feature Learning
- A synapse-centric account of the free energy principle
- Continuously Learning to Detect People on the Fly: A Bio-inspired Visual System for Drones
Cited by in corpus (5)
- Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing
- Applications of Spiking Neural Networks in Visual Place Recognition
- Energy efficiency analysis of Spiking Neural Networks for space applications
- The sounds of science a symphony for many instruments and voices part II
- Non-uniform Memory Partitioning For Low-Power Spiking Neural Networks