2 citations · 3 across the 10 of their papers we have counts for
8 papers · 1 filter
Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants
Yunhe Wang, Kai Han, Huiling Zhen +13
The paradigm of Large Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential ``brick-by-brick'' process. Despit…
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
Zhongqian Fu, Tianyi Zhao, Ning Ding +4
Mixture-of-Experts (MoE) models enable scalable computation and performance in large-scale deep learning but face quantization challenges due to sparse expert activation and dynami…
Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
Yehui Tang, Xiaosong Li, Fangcheng Liu +19
The surgence of Mixture of Experts (MoE) in Large Language Models promises a small price of execution cost for a much larger model parameter count and learning capacity, because on…
Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
Hanting Chen, Jiarui Qin, Jialong Guo +15
Large Language Models (LLMs) deliver state-of-the-art capabilities across numerous tasks, but their immense size and inference costs pose significant computational challenges for p…
Saliency-driven Dynamic Token Pruning for Large Language Models
Yao Tao, Yehui Tang, Yun Wang +3
Despite the recent success of large language models (LLMs), LLMs are particularly challenging in long-sequence inference scenarios due to the quadratic computational complexity of…
Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
Zhenni Bi, Kai Han, Chuanjian Liu +2
Large Language Models (LLMs) have demonstrated remarkable abilities across various language tasks, but solving complex reasoning problems remains a significant challenge. While exi…