3 citations · 5 across the 14 of their papers we have counts for
6 papers · 1 filter
AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
Ran Yan, Youhe Jiang, Tianyuan Wu +7
Maximizing training throughput and cost-efficiency of RL for LLMs is essential to democratize this advanced technique. One promising but challenging approach is to deploy such a co…
MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
Rizhen Hu, Yutong He, Ran Yan +3
As distributed optimization scales to meet the demands of Large Language Model (LLM) training, hardware failures become increasingly non-negligible. Existing fault-tolerant trainin…
UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
Guangxin He, Shen Nie, Fengqi Zhu +6
Diffusion LLMs have attracted growing interest, with plenty of recent work emphasizing their great potential in various downstream tasks; yet the long-context behavior of diffusion…
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
Ran Yan, Youhe Jiang, Zhuoming Chen +3
Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language model…
Oblivionis: A Lightweight Learning and Unlearning Framework for Federated Large Language Models
Fuyao Zhang, Xinyu Yan, Tiantong Wu +7
Large Language Models (LLMs) increasingly leverage Federated Learning (FL) to utilize private, task-specific datasets for fine-tuning while preserving data privacy. However, while…
HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment
Youhe Jiang, Ran Yan, Binhang Yuan
Disaggregating the prefill and decoding phases represents an effective new paradigm for generative inference of large language models (LLM), which eliminates prefill-decoding inter…