5 papers
Shaping Latent Mixture-of-Experts: From Expert Geometry to Layerwise Capacity Allocation
Yaoxiang Wang, Kai Zhang, Ruobing Xie +4
Fine-grained mixture-of-experts (MoE) models commonly increase expert count by narrowing each expert, treating expert shape as a by-product of routing granularity. LatentMoE moves…
AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding
Hong Liu, Rui Cen, Junhan Shi +10
Speculative decoding accelerates large language model inference without changing the target distribution, but no single drafting structure performs best across real-world workloads…
A Decomposition Perspective to Long-context Reasoning for LLMs
Yanling Xiao, Huaibing Xie, Guoliang Zhao +8
Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-c…
Self-Distillation for Multi-Token Prediction
Guoliang Zhao, Ruobing Xie, An Wang +3
As Large Language Models (LLMs) scale up, inference efficiency becomes a critical bottleneck. Multi-Token Prediction (MTP) could accelerate LLM inference by predicting multiple fut…
Probing How Scalable Table Data Enhances General Long-Context Reasoning
Huaibing Xie, Guoliang Zhao, Yang Liu +8
As real-world tasks grow increasingly complex, long-context reasoning has become a core capability for Large Language Models (LLMs). However, few studies explore which data types a…