3 citations · 3 across the 6 of their papers we have counts for
5 papers · 1 filter
ETH: Data-Balanced Heterogeneous Pipeline Parallelism for Training Large Models
Mianjie Yu, Zhongmin Zhao, Zizhao Mo +9
Current heterogeneity-aware training systems for large models balance workloads by assigning unequal numbers of layers across pipeline stages, a strategy aimed at equalizing execut…
Poseidon: DAG-Guided Parallelism Search for LLM Pre-Training on Heterogeneous Clusters
Xiaosong Chen, Shaoheng Nie, Zhongmin Zhao +6
With the rapid advancement of accelerator technologies, pre-training large language models (LLMs) on heterogeneous accelerator clusters has become increasingly crucial for maximizi…
psRL: Efficient Training for Agentic AI via Training-Time Prefix Sharing
Mianjie Yu, Zizhao Mo, Huanyu Qu +8
In modern agentic AI training, the system bottleneck is shifting from rollout to update. Emerging sampling strategies such as tree-structured and step-wise RL greatly increase trai…
Cloud-native and Distributed Systems for Efficient and Scalable Large Language Models -- A Research Agenda
Minxian Xu, Jingfeng Wu, Shengye Song +16
The rapid rise of Large Language Models (LLMs) has revolutionized various artificial intelligence (AI) applications, from natural language processing to code generation. However, t…
Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking
Zizhao Mo, Junlin Chen, Huanle Xu +1
Nowadays, service providers often deploy multiple types of LLM services within shared clusters. While the service colocation improves resource utilization, it introduces significan…