6 papers
Weight Group-wise Post-Training Quantization for Medical Foundation Model
Yineng Chen, Peng Huang, Aozhong Zhang +9
Foundation models have achieved remarkable results in medical image analysis. However, its large network architecture and high computational complexity significantly impact inferen…
Frayed RoPE and Long Inputs: A Geometric Perspective
Davis Wertheimer, Aozhong Zhang, Derrick Liu +2
Rotary Positional Embedding (RoPE) is a widely adopted technique for encoding position in language models, which, while effective, causes performance breakdown when input length ex…
DiaBlo: Diagonal Blocks Are Sufficient For Finetuning
Selcuk Gurses, Aozhong Zhang, Yanxia Deng +5
Fine-tuning is a critical step for adapting large language models (LLMs) to domain-specific downstream tasks. To mitigate the substantial computational and memory costs of full-mod…
CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization
Yanxia Deng, Aozhong Zhang, Selcuk Gurses +3
Fine-tuning large language models (LLMs) using low-rank adaptation (LoRA) has become a highly efficient approach for downstream tasks, particularly in scenarios with limited comput…
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
Aozhong Zhang, Naigang Wang, Yanxia Deng +3
In this paper, we present a simple optimization-based preprocessing technique called Weight Magnitude Reduction (MagR) to improve the performance of post-training quantization. For…
COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
Aozhong Zhang, Zi Yang, Naigang Wang +4
Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing t…