From the 1 of 14 linked papers with an AI index.
14 papers
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Zetao Hong, Song Yuan, Yuanhao Ding +4
Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. P…
TensorCast: The Missing Tensor Management Layer in Large Language Model Infrastructure
Yuhan Zhou, Yuchu Luo, Hao Nie +5
Modern LLM infrastructure increasingly manages tensors not only as computation data, but also as persistent states shared across distributed components. Existing systems optimize i…
Don't Predict, Prioritize: Rethinking GPU Reliability Assessment
Difeng Ma, Changhua Pei, Yuanwei Lu +7
The paper proposes HeaRank, a learning-to-rank framework that ranks GPU nodes by their relative failure risk instead of predicting exact failure times, showing improved detection o…
JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting
Lanxiang Hu, Zhaoxiang Feng, Yulun Wu +9
Speculative decoding (SD) accelerates autoregressive Large Language Models (LLMs) by drafting multiple tokens and verifying them in parallel, but it faces a scaling limitation: inc…
Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
Yicheng Feng, Xin Tan, Yangtao Deng +3
Modern LLM serving is no longer homogeneous or monolithic. Production systems now combine disaggregated execution, complex parallelism, runtime optimizations, and stateful workload…
StepAudio 2.5 Technical Report
Bin Lin, Bo Zhao, Boyong Wu +98
Unified audio-language modeling has emerged as a prominent trend in modern speech systems, promising to bring the reasoning capabilities of large language models to auditory tasks.…