Channel-Wise Mixed-Precision Quantization for Large Language Models

Zihan Chen, Bike Xie, Jundong Li +1

Large Language Models (LLMs) have demonstrated remarkable success across a wide range of language tasks, but their deployment on edge devices remains challenging due to the substan…