14 citations · 22 across the 7 of their papers we have counts for
4 papers · 1 filter
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
Dayou Du, Yijia Zhang, Shijie Cao +4
The upscaling of Large Language Models (LLMs) has yielded impressive advances in natural language processing, yet it also poses significant deployment challenges. Weight quantizati…
AFPQ: Asymmetric Floating Point Quantization for LLMs
Yijia Zhang, Sicheng Zhang, Shijie Cao +4
Large language models (LLMs) show great performance in various tasks, but face deployment challenges from limited memory capacity and bandwidth. Low-bit weight quantization can sav…
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Junyan Li, Li Lyna Zhang, Jiahang Xu +9
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with i…
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
Huiqiang Jiang, Li Lyna Zhang, Yuang Li +7
Automatic Speech Recognition (ASR) has seen remarkable advancements with deep neural networks, such as Transformer and Conformer. However, these models typically have large model s…