PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization

Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1

Mixture-of-Experts (MoE) is a popular class of large language models (LLMs), offering high efficiency and accuracy. However, in KV-cache-intensive serving scenarios, MoEs often exh…