37 citations · 154 across the 22 of their papers we have counts for
8 papers · 1 filter
Belayer: Efficient Fault Tolerance for LLM Agentic RL Training
Jiecheng Zhou, Qinghao Hu, Peng Sun +2
Large language model (LLM) agents are increasingly trained with reinforcement learning in long-horizon, sandboxed environments. Unlike conventional RL, agentic RL couples GPU-inten…
SpecGen: Accelerating Agentic Kernel Optimization with Speculative Generation
Jihu Guo, Sitian Lu, Tenghui Ma +4
Agentic kernel optimization automates manual GPU kernel tuning via iterative generation, validation, and profiling with reasoning LLMs, casting the optimization task as feedback-gu…
Tropical: Enhancing SLO Attainment in Disaggregated LLM Serving via SLO-Aware Multiplexing
Jinming Ma, Jiefei Chen, Xiuhong Li +5
To guarantee service quality in transformer based large language model (LLM) serving, it is essential to meet the latency constraints of both the prefill phase (measured by Time-to…
NanoCP: Taming Load Imbalance in Data-Expert Parallel Decoding under Variable-Length Workloads
Jiefei Chen, Binbin Lin, Jinming Ma +9
Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data pa…
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
Jiangfei Duan, Shuo Zhang, Zerui Wang +13
Large Language Models (LLMs) like GPT and LLaMA are revolutionizing the AI industry with their sophisticated capabilities. Training these models requires vast GPU clusters and sign…
MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
Jiangfei Duan, Runyu Lu, Haojie Duanmu +5
Large language models (LLMs) have demonstrated remarkable performance, and organizations are racing to serve LLMs of varying sizes as endpoints for use-cases like chat, programming…
ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
Ding Tang, Lijuan Jiang, Jiecheng Zhou +5
Large-scale models rely heavily on 3D parallelism for distributed training, which utilizes tensor parallelism (TP) as the intra-operator parallelism to partition model states acros…
Proteus: Simulating the Performance of Distributed DNN Training
Jiangfei Duan, Xiuhong Li, Ping Xu +4
DNN models are becoming increasingly larger to achieve unprecedented accuracy, and the accompanying increased computation and memory requirements necessitate the employment of mass…