8 citations · 17 across the 6 of their papers we have counts for
3 papers · 1 filter
Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMs
Jung Hyun Lee, Seungjae Shin, Vinnam Kim +2
As the rapid scaling of large language models (LLMs) poses significant challenges for deployment on resource-constrained devices, there is growing interest in extremely low-bit qua…
How to Parameterize Asymmetric Quantization Ranges for Quantization-Aware Training
Jaeseong You, Minseop Park, Kyunggeun Lee +3
This paper investigates three different parameterizations of asymmetric uniform quantization for quantization-aware training: (1) scale and offset, (2) minimum and maximum, and (3)…
Quadapter: Adapter for GPT-2 Quantization
Minseop Park, Jaeseong You, Markus Nagel +1
Transformer language models such as GPT-2 are difficult to quantize because of outliers in activations leading to a large quantization error. To adapt to the error, one must use qu…