Post-Training BatchNorm Recalibration
arXiv:2010.05625
Abstract
We revisit non-blocking simultaneous multithreading (NB-SMT) introduced previously by Shomron and Weiser (2020). NB-SMT trades accuracy for performance by occasionally "squeezing" more than one thread into a shared multiply-and-accumulate (MAC) unit. However, the method of accommodating more than one thread in a shared MAC unit may contribute noise to the computations, thereby changing the internal statistics of the model. We show that substantial model performance can be recouped by post-training recalibration of the batch normalization layers' running mean and running variance statistics, given the presence of NB-SMT.
References in corpus (11)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- cuDNN: Efficient Primitives for Deep Learning
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Improving robustness against common corruptions by covariate shift adaptation
- Post-training 4-bit quantization of convolution networks for rapid-deployment
- Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
- Four Things Everyone Should Know to Improve Batch Normalization
- Robust Quantization: One Model to Rule Them All
- Robust Processing-In-Memory Neural Networks via Noise-Aware Normalization
- Non-Blocking Simultaneous Multithreading: Embracing the Resiliency of Deep Neural Networks