Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference

Abstract

Efficient machine learning implementations optimized for inference in hardware have wide-ranging benefits, depending on the application, from lower inference latency to higher data throughput and reduced energy consumption. Two popular techniques for reducing computation in neural networks are pruning, removing insignificant synapses, and quantization, reducing the precision of the calculations. In this work, we explore the interplay between pruning and quantization during the training of neural networks for ultra low latency applications targeting high energy physics use cases. Techniques developed for this study have potential applications across many other domains. We study various configurations of pruning during quantization-aware training, which we term quantization-aware pruning, and the effect of techniques like regularization, batch normalization, and different pruning schemes on performance, computational complexity, and information content metrics. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task. Further, quantization-aware pruning typically performs similar to or better in terms of computational efficiency compared to other neural architecture search techniques like Bayesian optimization. Surprisingly, while networks with different training configurations can have similar performance for the benchmark application, the information content in the network can vary significantly, affecting its generalizability.

arXiv comment22 pages, 7 Figures, 1 Table

References in corpus (8)

• Fast inference of deep neural networks in FPGAs for particle physics
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• JEDI-net: a jet identification algorithm based on interaction networks
• Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML
• UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks
• Bayesian Bits: Unifying Quantization and Pruning
• The importance of calorimetry for highly-boosted jet substructure
• HCM: Hardware-Aware Complexity Metric for Neural Network Architectures

Cited by in corpus (9)

• Applications and Techniques for Fast Machine Learning in Science
• Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation
• Charged particle tracking via edge-classifying interaction networks
• Graph Neural Networks for Charged Particle Tracking on FPGAs
• Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics
• Symbolic Regression on FPGAs for Fast Machine Learning Inference
• Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics
• BitHEP -- The Limits of Low-Precision ML in HEP
• From Tiny Machine Learning to Tiny Deep Learning: A Survey