4 papers
Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
Long Zhao, Qinghe Wang, Jiaan Zhu +5
Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality. However, the synchronous three-stage RLHF pipeline is often b…
AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Network
Chaoyi Ruan, Geng Luo, Xinyi Wan +12
LLM reinforcement learning (RL) requires frequent synchronization of large model parameters between the trainer and distributed rollout actors. High-throughput RL post-training the…
LiteCache: A Query Similarity-Driven, GPU-Centric KVCache Subsystem for Efficient LLM Inference
Jiawei Yi, Ping Gong, Youhui Bai +10
During LLM inference, KVCache memory usage grows linearly with sequence length and batch size and often exceeds GPU capacity. Recent proposals offload KV states to host memory and…
BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference
Zewen Jin, Shengnan Wang, Jiaan Zhu +5
The Mixture-of-Experts (MoE) structure scales the Transformer-based large language models (LLMs) and improves their performance with only the sub-linear increase in computation res…