5 papers
SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning
Bo Liu, Leon Guertler, Simon Yu +9
Recent advances in reinforcement learning have shown that language models can develop sophisticated reasoning through training on tasks with verifiable rewards, but these approache…
Revisiting Parameter Server in LLM Post-Training
Xinyi Wan, Penghui Qi, Guangxing Huang +3
Modern data parallel (DP) training favors collective communication over parameter servers (PS) for its simplicity and efficiency under balanced workloads. However, the balanced wor…
PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
Xinyi Wan, Penghui Qi, Guangxing Huang +2
Pipeline parallelism (PP) is widely used for training large language models (LLMs), yet its scalability is often constrained by high activation memory consumption as the number of…
Balancing Pipeline Parallelism with Vocabulary Parallelism
Man Tsung Yeung, Penghui Qi, Min Lin +1
Pipeline parallelism is widely used to scale the training of transformer-based large language models, various works have been done to improve its throughput and memory footprint. I…
Pipeline Parallelism with Controllable Memory
Penghui Qi, Xinyi Wan, Nyamdavaa Amar +1
Pipeline parallelism has been widely explored, but most existing schedules lack a systematic methodology. In this paper, we propose a framework to decompose pipeline schedules as r…