Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM

Luoming Zhang, Wen Fei, Weijia Wu +3

Large Language Models (LLMs) pose significant hardware challenges related to memory requirements and computational ability. There are two mainstream quantization schemes for LLMs:…