1 citations · 3 across the 27 of their papers we have counts for
6 papers · 1 filter
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Zetao Hong, Song Yuan, Yuanhao Ding +4
Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. P…
TensorCast: The Missing Tensor Management Layer in Large Language Model Infrastructure
Yuhan Zhou, Yuchu Luo, Hao Nie +5
Modern LLM infrastructure increasingly manages tensors not only as computation data, but also as persistent states shared across distributed components. Existing systems optimize i…
Don't Predict, Prioritize: Rethinking GPU Reliability Assessment
Difeng Ma, Changhua Pei, Yuanwei Lu +7
The reliability of Graphics Processing Units (GPUs) is a criticalbottleneck for modern large-scale AI infrastructure, where a sin-gle node failure can disrupt synchronous training…
DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
Zhenliang Xue, Hanpeng Hu, Xing Chen +7
Large multimodal models (LMMs) have demonstrated excellent capabilities in both understanding and generation tasks with various modalities. While these models can accept flexible c…
DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training
Xin Tan, Yuetao Chen, Yimin Jiang +6
Diffusion Transformers (DiTs) have shown remarkable performance in generating high-quality videos. However, the quadratic complexity of 3D full attention remains a bottleneck in sc…
DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
Zili Zhang, Yinmin Zhong, Yimin Jiang +6
Multimodal large language models (LLMs) empower LLMs to ingest inputs and generate outputs in multiple forms, such as text, image, and audio. However, the integration of multiple m…