6 papers
TAOT: Topology-Aware Optimal Transport for Dynamic Expert Replica Placement in MoE Training
Lingyun Zhang, Henghua Zhang, Shilei Gu +5
Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs), yet its dynamic routing causes severe load imbalance in expert-parallel training. E…
CVPO: Enhancing LLM Reinforcement Learning Reasoning via Value-Variance Adaptation and Dynamic Curriculum Learning
Ziqi Jia, Yalu Ouyang, Bo Pang +5
Reinforcement learning (RL) has emerged as an effective method for enhancing the reasoning capabilities of large language models (LLMs). However, existing methods suffer from insuf…
Predicting Future Utility: Global Combinatorial Optimization for Task-Agnostic KV Cache Eviction
Ziyao Tang, Pengkun Jiao, Xinhang Chen +3
Given the quadratic complexity of attention, KV cache eviction is vital to accelerate model inference. Current KV cache eviction methods typically rely on instantaneous heuristic m…
FEVO: Financial Knowledge Expansion and Reasoning Evolution for Large Language Models
Bo Pang, Yalu Ouyang, Hangfei Xu +6
Advancements in reasoning for large language models (LLMs) have lead to significant performance improvements for LLMs in various fields such as mathematics and programming. However…
Astra: Efficient and Money-saving Automatic Parallel Strategies Search on Heterogeneous GPUs
Peiran Wang, Haibing Li, Fu Haohan +3
In this paper, we introduce an efficient and money-saving automatic parallel strategies search framework on heterogeneous GPUs: Astra. First, Astra searches for the efficiency-opti…
Mao: Machine learning approach for NUMA optimization in Warehouse Scale Computers
Yueji Liu, Jun Jin, Wenhui Shu +2
Non-Uniform Memory Access (NUMA) architecture imposes numerous performance challenges to today's cloud workloads. Due to the complexity and the massive scale of modern warehouse-sc…