On the Compressibility of Quantized Large Language Models

Yu Mao, Weilan Wang, Hongchao Du +2

Deploying Large Language Models (LLMs) on edge or mobile devices offers significant benefits, such as enhanced data privacy and real-time processing capabilities. However, it also…