Any-Precision Deep Neural Networks

Abstract

We present any-precision deep neural networks (DNNs), which are trained with a new method that allows the learned DNNs to be flexible in numerical precision during inference. The same model in runtime can be flexibly and directly set to different bit-widths, by truncating the least significant bits, to support dynamic speed and accuracy trade-off. When all layers are set to low-bits, we show that the model achieved accuracy comparable to dedicated models trained at the same precision. This nice property facilitates flexible deployment of deep learning models in real-world applications, where in practice trade-offs between model accuracy and runtime efficiency are often sought. Previous literature presents solutions to train models at each individual fixed efficiency/accuracy trade-off point. But how to produce a model flexible in runtime precision is largely unexplored. When the demand of efficiency/accuracy trade-off varies from time to time or even dynamically changes in runtime, it is infeasible to re-train models accordingly, and the storage budget may forbid keeping multiple models. Our proposed framework achieves this flexibility without performance degradation. More importantly, we demonstrate that this achievement is agnostic to model architectures and applicable to multiple vision tasks. Our code is released at https://github.com/SHI-Labs/Any-Precision-DNNs.

arXiv commentAAAI 2021

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Rethinking Atrous Convolution for Semantic Image Segmentation
• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Slimmable Neural Networks
• Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
• SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision

Cited by in corpus (5)

• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis
• BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
• Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
• Dynamic Network Quantization for Efficient Video Inference
• One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustment