Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving

Abstract

Bayesian Neural Networks (BNNs) that possess a property of uncertainty estimation have been increasingly adopted in a wide range of safety-critical AI applications which demand reliable and robust decision making, e.g., self-driving, rescue robots, medical image diagnosis. The training procedure of a probabilistic BNN model involves training an ensemble of sampled DNN models, which induces orders of magnitude larger volume of data movement than training a single DNN model. In this paper, we reveal that the root cause for BNN training inefficiency originates from the massive off-chip data transfer by Gaussian Random Variables (GRVs). To tackle this challenge, we propose a novel design that eliminates all the off-chip data transfer by GRVs through the reversed shifting of Linear Feedback Shift Registers (LFSRs) without incurring any training accuracy loss. To efficiently support our LFSR reversion strategy at the hardware level, we explore the design space of the current DNN accelerators and identify the optimal computation mapping scheme to best accommodate our strategy. By leveraging this finding, we design and prototype the first highly efficient BNN training accelerator, named Shift-BNN, that is low-cost and scalable. Extensive evaluation on five representative BNN models demonstrates that Shift-BNN achieves an average of 4.9x (up to 10.8x) boost in energy efficiency and 1.6x (up to 2.8x) speedup over the baseline DNN training accelerator.

54th IEEE/ACM International Symposium on Microarchitecture

References in corpus (17)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Variational Inference: A Review for Statisticians
• End to End Learning for Self-Driving Cars
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Deep Learning with Limited Numerical Precision
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• Scalable Methods for 8-bit Training of Neural Networks
• SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
• A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference
• Distributed Deep Learning Using Synchronous Stochastic Gradient Descent
• VIBNN: Hardware Acceleration of Bayesian Neural Networks
• TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and Inference
• All-Spin Bayesian Neural Networks
• FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
• Spatial Uncertainty Sampling for End-to-End Control
• On Machine Learning and Structure for Mobile Robots