4 papers
Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +4
Large language models require significant computational resources for deployment, making quantization essential for practical applications. However, the main obstacle to effective…
Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +3
Large language models (LLMs) have significantly advanced natural language processing, but their massive parameter counts create substantial computational and memory challenges duri…
Sharpness-Aware Data Generation for Zero-shot Quantization
Dung Hoang-Anh, Cuong Pham Trung Le, Jianfei Cai +1
Zero-shot quantization aims to learn a quantized model from a pre-trained full-precision model with no access to original real training data. The common idea in zero-shot quantizat…
MetaAug: Meta-Data Augmentation for Post-Training Quantization
Cuong Pham, Hoang Anh Dung, Cuong C. Nguyen +4
Post-Training Quantization (PTQ) has received significant attention because it requires only a small set of calibration data to quantize a full-precision model, which is more pract…