11 papers
Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
Long Zhao, Qinghe Wang, Jiaan Zhu +5
Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality. However, the synchronous three-stage RLHF pipeline is often b…
HCT-QA: A Benchmark for Question Answering on Human-Centric Tables
Mohammad S. Ahmad, Zan A. Naeem, Michaël Aupetit +6
Tabular data embedded in PDF files, web pages, and other types of documents is prevalent in various domains. These tables, which we call human-centric tables (HCTs for short), are…
Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
Guanbin Xu, ZhenGuo Xu, Yuzhe Li +4
Overlapping communication with computation is crucial for distributed large-model training, yet optimizing it - especially when computation becomes the bottleneck-remains challengi…
AuroraRL: Fast, Fault-Tolerant, and Cost-Efficient Reinforcement Learning over Decentralized Network
Chaoyi Ruan, Geng Luo, Xinyi Wan +12
LLM reinforcement learning (RL) requires frequent synchronization of large model parameters between the trainer and distributed rollout actors. High-throughput RL post-training the…
Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
Chaoyi Ruan, Chao Bi, Kaiwen Zheng +3
Large Language Model (LLM) agents tackle data-intensive tasks such as deep research and code generation. However, their effectiveness depends on frequent interactions with knowledg…
Revisiting Parameter Server in LLM Post-Training
Xinyi Wan, Penghui Qi, Guangxing Huang +3
Modern data parallel (DP) training favors collective communication over parameter servers (PS) for its simplicity and efficiency under balanced workloads. However, the balanced wor…