37 citations · 154 across the 22 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025★ 1 cited
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
Haojie Duanmu, Xiuhong Li, Zhihang Yuan +4
Mixture-of-Experts (MoE) models face deployment challenges due to their large parameter counts and computational demands. We explore quantization for MoE models and highlight two k…
cs.LG2024
SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
Haojie Duanmu, Zhihang Yuan, Xiuhong Li +3
Large language models (LLMs) can now handle longer sequences of tokens, enabling complex tasks like book understanding and generating lengthy novels. However, the key-value (KV) ca…
cs.LG2022★ 6 cited
DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation
Yu Tang, Chenyu Wang, Yufan Zhang +5
The further development of deep neural networks is hampered by the limited GPU memory resource. Therefore, the optimization of GPU memory resources is highly demanded. Swapping and…