Mixed-Precision Graph Neural Quantization for Low Bit Large Language Models

Wanlong Liu, Yichen Xiao, Dingyi Zeng +3

Post-Training Quantization (PTQ) is pivotal for deploying large language models (LLMs) within resource-limited settings by significantly reducing resource demands. However, existin…