14 papers
Accelerating MoE Training through Expert Placement and Replica Coordination
Zhiqiang Tan, Maoxin Wang, Wenxiang Lin +4
Distributed training of sparsely activated mixture-of-experts (MoE) models is often bottlenecked by workload imbalance and all-to-all (A2A) communication induced by dynamic token r…
LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL
Sijie Wang, Zhiqiang Tan, Xinrui Yang +1
Diffusion reinforcement learning (RL) has recently achieved significant success in post-training image and video generative models. However, most diffusion RL methods, including Da…
Laplacian Frequency Hierarchies for Efficient 3D Gaussian Splatting Training
Yixiong Yang, Sisheng Zhang, Qingsong Yan +2
A key bottleneck in 3D Gaussian Splatting training is the continual growth of Gaussian primitives, which increases optimization cost and slows convergence, especially at high resol…
Compass: Dissecting Communication and Computation Operators for Efficient LLM Training
Guangyu Xiang, Lin Zhang, Haoxuan Yu +3
Overlapping communication and computation operators is a common practice to hide communication overheads, accelerating large language models (LLMs) training on GPU clusters. Existi…
Bidirectional Resource Scheduling for Disaggregated and Asynchronous RL Post-Training
Zhiqiang Tan, Maoxin Wang, Sijie Wang +4
It is well established that the reasoning capabilities of large language models (LLMs) can be improved by applying reinforcement learning (RL) in a post-training stage. In a standa…
Zellige: Moldable Sequence Placement for Mixed Image-Video DiT Training
Guangyu Xiang, Xueze Kang, Minwei Zhao +4
High-quality video generation requires training Diffusion Transformers (DiTs) jointly on image and video data, posing a mixed-length sequence training problem across GPUs. Existing…
Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration
Xueze Kang, Guangyu Xiang, Suyi Li +4
Diffusion models are increasingly deployed as production visual-generation services, where serving high-resolution image and long video generation is often limited by GPU memory. P…
KernelFlume: Elastic Core-Attention Scaling for Agentic Long-Context Decoding
Guangyu Xiang, Xueze Kang, Lin Zhang +4
LLM serving is increasingly dominated by long and dynamic decode workloads from agents, reasoning models, and extended conversations. When bursty long-context demand exceeds deploy…
AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
Wenxiang Lin, Juntao Huang, Luhan Zhang +5
Quantization is a key method for reducing the GPU memory requirement of training large language models (LLMs). Yet, current approaches are ineffective for 4-bit activations and 8-b…
Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
Xinglin Pan, Shaohuai Shi, Wenxiang Lin +4
The mixture-of-experts (MoE) architecture scales model size with sublinear computational increase but suffers from memory-intensive inference due to KV caches and sparse expert act…
Janus: Disaggregating Attention and Experts for Scalable MoE Inference
Zhexiang Zhang, Ye Wang, Yumiao Zhao +9
Serving large Mixture-of-Experts (MoE) models is challenging because of their large memory footprints, heterogeneous resource demands, and highly dynamic inference workloads. Most…
HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
Wenxiang Lin, Xinglin Pan, Lin Zhang +3
The sparsely activated mixture-of-experts (MoE) transformer has become a common architecture for large language models (LLMs) due to its sparsity, which requires fewer computationa…
Task Scheduling for Efficient Inference of Large Language Models on Single Moderate GPU Systems
Wenxiang Lin, Xinglin Pan, Shaohuai Shi +2
Large language models~(LLMs) are known for their high demand on computing resources and memory due to their substantial model size, which leads to inefficient inference on moderate…
Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing
Yizhou Luo, Qiang Wang, Shaohuai Shi +4
Deep learning (DL) has demonstrated significant success across diverse fields, leading to the construction of dedicated GPU accelerators within GPU clusters for high-quality traini…