most citedBitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation

1 citations · 1 across the 2 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2024★ 1 cited

BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation

Dayou Du, Yijia Zhang, Shijie Cao +4

The upscaling of Large Language Models (LLMs) has yielded impressive advances in natural language processing, yet it also poses significant deployment challenges. Weight quantizati…

cs.CL2023

AFPQ: Asymmetric Floating Point Quantization for LLMs

Yijia Zhang, Sicheng Zhang, Shijie Cao +4

Large language models (LLMs) show great performance in various tasks, but face deployment challenges from limited memory capacity and bandwidth. Low-bit weight quantization can sav…