Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence

Abstract

While Moore's law has driven exponential computing power expectations, its nearing end calls for new avenues for improving the overall system performance. One of these avenues is the exploration of alternative brain-inspired computing architectures that aim at achieving the flexibility and computational efficiency of biological neural processing systems. Within this context, neuromorphic engineering represents a paradigm shift in computing based on the implementation of spiking neural network architectures in which processing and memory are tightly co-located. In this paper, we provide a comprehensive overview of the field, highlighting the different levels of granularity at which this paradigm shift is realized and comparing design approaches that focus on replicating natural intelligence (bottom-up) versus those that aim at solving practical artificial intelligence applications (top-down). First, we present the analog, mixed-signal and digital circuit design styles, identifying the boundary between processing and memory through time multiplexing, in-memory computation, and novel devices. Then, we highlight the key tradeoffs for each of the bottom-up and top-down design approaches, survey their silicon implementations, and carry out detailed comparative analyses to extract design guidelines. Finally, we identify necessary synergies and missing elements required to achieve a competitive advantage for neuromorphic systems over conventional machine-learning accelerators in edge computing applications, and outline the key ingredients for a framework toward neuromorphic intelligence.

Accepted for publication in Proceedings of the IEEE

References in corpus (29)

• Deep Learning in Neural Networks: An Overview
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Theoretical Models of Learning to Learn
• DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
• A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)
• Brain-inspired computing: We need a master plan
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey
• Direct Feedback Alignment Provides Learning in Deep Neural Networks
• MorphIC: A 65-nm 738k-Synapse/mm${}^2$ Quad-Core Binary-Weight Digital Neuromorphic Processor with Stochastic Spike-Driven Online Learning
• Adaptive Extreme Edge Computing for Wearable Devices
• ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales
• Biologically Inspired Spiking Neurons : Piecewise Linear Models and Digital Implementation
• An electronic neuromorphic system for real-time detection of High Frequency Oscillations (HFOs) in intracranial EEG
• Training Neural Networks with Local Error Signals
• Real-time ultra-low power ECG anomaly detection using an event-driven neuromorphic processor
• SpiNNaker 2: A 10 Million Core Processor System for Brain Simulation and Machine Learning
• Low-Power Low-Latency Keyword Spotting and Adaptive Control with a SpiNNaker 2 Prototype and Comparison with Loihi
• Closed-loop spiking control on a neuromorphic processor implemented on the iCub
• RANC: Reconfigurable Architecture for Neuromorphic Computing
• Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
• Understanding Synthetic Gradients and Decoupled Neural Interfaces
• Principled Training of Neural Networks with Direct Feedback Alignment
• EDEN: A high-performance, general-purpose, NeuroML-based neural simulator
• Equilibrium Propagation with Continual Weight Updates
• NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems
• Disentanglement with Biological Constraints: A Theory of Functional Cell Types
• A New Look at Spike-Timing-Dependent Plasticity Networks for Spatio-Temporal Feature Learning
• A synapse-centric account of the free energy principle
• Continuously Learning to Detect People on the Fly: A Bio-inspired Visual System for Drones

Cited by in corpus (5)

• Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing
• Applications of Spiking Neural Networks in Visual Place Recognition
• Energy efficiency analysis of Spiking Neural Networks for space applications
• The sounds of science a symphony for many instruments and voices part II
• Non-uniform Memory Partitioning For Low-Power Spiking Neural Networks