9 papers
SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling
Weiqiao Shan, Ruixiang Mao, Yuang Li +10
Mixture-of-Experts (MoE) models enable efficient scaling, but training them from scratch remains prohibitively expensive. MoE upcycling mitigates this cost by converting pretrained…
Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation
Zhongzhu Zhou, Qingyang Wu, Junxiong Wang +4
Hybrid linear attention models offer an appealing path to faster long-context inference: they reduce the quadratic cost and KV-cache burden of full softmax attention while retainin…
NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Jiangyong Yu, Xiaomeng Han, Xing Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and c…
DLLMQuant: Quantizing Diffusion-based Large Language Models
Chen Xu, Dawei Yang
Diffusion-based large language models (DLLMs) have shown promise for non-autoregressive text generation, but their deployment is constrained by large model sizes and heavy computat…
MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization
JiangYong Yu, Sifan Zhou, Dawei Yang +7
Multimodal large language models (MLLMs) have garnered widespread attention due to their ability to understand multimodal input. However, their large parameter sizes and substantia…
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
Xing Hu, Zhixuan Chen, Dawei Yang +5
Mixture-of-Experts (MoE) large language models (LLMs), which leverage dynamic routing and sparse activation to enhance efficiency and scalability, have achieved higher performance…