LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment

Binrui Zeng, Bin Ji, Xiaodong Liu +7

As Large Language Models (LLMs) demonstrate exceptional performance across various domains, deploying LLMs on edge devices has emerged as a new trend. Quantization techniques, whic…