VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

Abstract

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.

11 pages, 12 figures

References in corpus (1)

• VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

Cited by in corpus (26)

• VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing
• p-Bits for Probabilistic Spin Logic
• Accelerating CNN inference on FPGAs: A Survey
• Energy-efficient stochastic computing with superparamagnetic tunnel junctions
• Low Barrier Magnet Design for Efficient Hardware Binary Stochastic Neurons
• Dielectrics for Two-Dimensional Transition Metal Dichalcogenide Applications
• Low-Energy Deep Belief Networks using Intrinsic Sigmoidal Spintronic-based Probabilistic Neurons
• Composable Probabilistic Inference Networks Using MRAM-based Stochastic Neurons
• A Stochastic-Computing based Deep Learning Framework using Adiabatic Quantum-Flux-Parametron SuperconductingTechnology
• Memory-Efficient FPGA Implementation of Stochastic Simulated Annealing
• Enhanced Convergence in p-bit Based Simulated Annealing with Partial Deactivation for Large-Scale Combinatorial Optimization Problems
• ThUnderVolt: Enabling Aggressive Voltage Underscaling and Timing Error Resilience for Energy Efficient Deep Neural Network Accelerators
• Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations
• A Study of Deep Learning Robustness Against Computation Failures
• Sigma Delta Quantized Networks
• Fast Solving Complete 2000-Node Optimization Using Stochastic-Computing Simulated Annealing
• Local Energy Distribution Based Hyperparameter Determination for Stochastic Simulated Annealing
• Stochastic Simulated Quantum Annealing for Fast Solution of Combinatorial Optimization Problems
• High Convergence Rates of CMOS Invertible Logic Circuits Based on Many-Body Hamiltonians
• DEMOTIC: A Differentiable Sampler for Multi-Level Digital Circuits
• Hardware-Driven Nonlinear Activation for Stochastic Computing Based Deep Convolutional Neural Networks
• Energy-Efficient p-Bit-Based Fully-Connected Quantum-Inspired Simulated Annealer with Dual BRAM Architecture
• From DNNs to GANs: Review of efficient hardware architectures for deep learning
• Optical Stochastic Computing Architectures Using Photonic Crystal Nanocavities
• SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks
• Stochastic Computing for Hardware Implementation of Binarized Neural Networks