3 papers
cs.LG2026
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Yuanteng Chen, Peisong Wang, Zhilei Liu +9
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods…
cs.LG2026
Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE
Yuanteng Chen, Peisong Wang, Nanxin Zeng +5
Test-time scaling improves LLM performance by generating multiple candidate solutions, yet token-level sampling requires temperature tuning that trades off diversity against stabil…
cs.LG2025
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
Yuanteng Chen, Peisong Wang, Yuantian Shao +3
Sparse Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs) efficiently. Recent fine-grained MoE designs introduce hundreds of experts pe…