81 citations · 344 across the 26 of their papers we have counts for
7 papers · 1 filter
Quadapter: Adapter for GPT-2 Quantization
Minseop Park, Jaeseong You, Markus Nagel +1
Transformer language models such as GPT-2 are difficult to quantize because of outliers in activations leading to a large quantization error. To adapt to the error, one must use qu…
FP8 Quantization: The Power of the Exponent
Andrey Kuzmin, Mart Van Baalen, Yuwei Ren +3
When quantizing neural networks for efficient inference, low-bit integers are the go-to format for efficiency. However, low-bit floating point numbers have an extra degree of freed…
Quantized Sparse Weight Decomposition for Neural Network Compression
Andrey Kuzmin, Mart van Baalen, Markus Nagel +1
In this paper, we introduce a novel method of neural network weight compression. In our method, we store weight tensors as sparse, quantized matrix factors, whose product is comput…
Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices
Kartik Gupta, Marios Fournarakis, Matthias Reisser +2
Federated Learning (FL) is a machine learning paradigm to distributively learn machine learning models from decentralized data that remains on-device. Despite the success of standa…
Overcoming Oscillations in Quantization-Aware Training
Markus Nagel, Marios Fournarakis, Yelysei Bondarenko +1
When training neural networks with simulated quantization, we observe that quantized weights can, rather unexpectedly, oscillate between two grid-points. The importance of this eff…
Cyclical Pruning for Sparse Neural Networks
Suraj Srinivas, Andrey Kuzmin, Markus Nagel +3
Current methods for pruning neural network weights iteratively apply magnitude-based pruning on the model weights and re-train the resulting model to recover lost accuracy. In this…
Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)
Sangeetha Siddegowda, Marios Fournarakis, Markus Nagel +3
While neural networks have advanced the frontiers in many machine learning applications, they often come at a high computational cost. Reducing the power and latency of neural netw…