LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

Zechun Liu, Barlas Oguz, Changsheng Zhao +6

Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits. We find that these methods break d…