QPyTorch: A Low-Precision Arithmetic Simulation Framework

Abstract

Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to empirically evaluate the statistical effects of quantization while avoiding the substantial overhead of building specific hardware. To support this empirical research, we introduce QPyTorch, a low-precision arithmetic simulation framework. Built natively in PyTorch, QPyTorch provides a convenient interface that minimizes the efforts needed to reliably convert existing codes to study low-precision training. QPyTorch is general, and supports a variety of combinations of precisions, number formats, and rounding options. Additionally, it leverages an efficient fused-kernel approach to reduce simulator overhead, which enables simulation of large-scale, realistic problems. QPyTorch is publicly available at https://github.com/Tiiiger/QPyTorch.

NeurIPS 2019 EMC^2 Workshop on Energy Efficient Machine Learning and Cognitive Computing

References in corpus (5)

• TensorFlow: A system for large-scale machine learning
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Binarized Neural Networks
• Training Deep Neural Networks with 8-bit Floating Point Numbers
• Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point

Cited by in corpus (4)

• Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision
• Revisiting BFloat16 Training
• Low-Complexity LSTM Training and Inference with FloatSD8 Weight Representation
• Empirical Evaluation of Deep Learning Model Compression Techniques on the WaveNet Vocoder