Choose your tools carefully: A Comparative Evaluation of Deterministic vs. Stochastic and Binary vs. Analog Neuron models for Implementing Emerging Computing Paradigms

Abstract

Neuromorphic computing, commonly understood as a computing approach built upon neurons, synapses, and their dynamics, as opposed to Boolean gates, is gaining large mindshare due to its direct application in solving current and future computing technological problems, such as smart sensing, smart devices, self-hosted and self-contained devices, artificial intelligence (AI) applications, etc. In a largely software-defined implementation of neuromorphic computing, it is possible to throw enormous computational power or optimize models and networks depending on the specific nature of the computational tasks. However, a hardware-based approach needs the identification of well-suited neuronal and synaptic models to obtain high functional and energy efficiency, which is a prime concern in size, weight, and power (SWaP) constrained environments. In this work, we perform a study on the characteristics of hardware neuron models (namely, inference errors, generalizability and robustness, practical implementability, and memory capacity) that have been proposed and demonstrated using a plethora of emerging nano-materials technology-based physical devices, to quantify the performance of such neurons on certain classes of problems that are of great importance in real-time signal processing like tasks in the context of reservoir computing. We find that the answer on which neuron to use for what applications depends on the particulars of the application requirements and constraints themselves, i.e., we need not only a hammer but all sorts of tools in our tool chest for high efficiency and quality neuromorphic computing.

arXiv comment13 pages, 6 figures

References in corpus (14)

• Recent Advances in Physical Reservoir Computing: A Review
• Photonics for artificial intelligence and neuromorphic computing
• Spin torque building blocks
• Magnetic skyrmion artificial synapse for neuromorphic computing
• Stochastic p-bits for Invertible Logic
• Understanding the dynamics of biological and neural oscillator networks through exact mean-field reductions: a review
• Superconducting optoelectronic circuits for neuromorphic computing
• Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks
• Magnetic domain wall based synaptic and activation function generator for neuromorphic accelerators
• Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation
• Using reservoir computer to predict and prevent extreme events
• A Variational Perspective on Diffusion-Based Generative Models and Score Matching
• On the role of synaptic stochasticity in training low-precision neural networks
• Analog Signal Processing Using Stochastic Magnets