Publications (19)
GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
Jiaang Duan, Shenglin Xu, Shiyou Qian +15
The surge in large language models (LLMs) has fundamentally reshaped the landscape of GPU usage patterns, creating an urgent need for more efficient management strategies. While cl…
Diving into 3D Parallelism with Heterogeneous Spot Instance GPUs: Design and Implications
Yuxiao Wang, Yuedong Xu, Qingyang Duan +4
The rapid growth of large language models (LLMs) and the continuous release of new GPU products have significantly increased the demand for distributed training across heterogeneou…
Bifrost: Empowering Pretrained Language Model with Fallibility Representation for Log-Based Fault Diagnosis
Minghua He, Tong Jia, Lingzhe Zhang +9
Log-based fault diagnosis is crucial for runtime debugging and maintenance. Existing fault diagnosis methods use language models pre-trained on natural language (PLMs) for log repr…
Flow Level QoE of Video Streaming in Wireless Networks
Yuedong Xu, Salaheddine Elayoubi, Eitan Altman +2
The Quality of Experience (QoE) of streaming service is often degraded by frequent playback interruptions. To mitigate the interruptions, the media player prefetches streaming cont…
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
Lingyun Yang, Yuxiao Wang, Shenghao Liang +8
The paper introduces Atrex-Bench, a trace-driven GPU kernel benchmark derived from real production inference workloads, and evaluates LLM-generated kernels, revealing a large perfo…
WorldOlympiad: Can Your World Model Survive a Triathlon?
Yuke Zhao, Wangbo Zhao, Weijie Wang +8
We introduce WorldOlympiad, a benchmark for diagnosing video-based world models across physical faithfulness, geometric consistency, and interaction fidelity. While existing benchm…
Adaptra: Straggler-Resilient Hybrid-Parallel Training with Pipeline Adaptation
Tianyuan Wu, Lunxi Cao, Hanfeng Lu +8
Training large Deep Neural Network (DNN) models at scale often encounters straggler issues, mostly in communications due to network congestion, RNIC/switch defects, or topological…
InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling
Xiaoxiao Jiang, Suyi Li, Lingyun Yang +12
Generative image editing using diffusion models has become a prevalent application in today's AI cloud services. In production environments, image editing typically involves a mask…
FALCON: Pinpointing and Mitigating Stragglers for Large-Scale Hybrid-Parallel Training
Tianyuan Wu, Wei Wang, Yinghao Yu +7
Fail-slows, or stragglers, are common but largely unheeded problems in large-scale hybrid-parallel training that spans thousands of GPU servers and runs for weeks to months. Yet, t…
LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows
Lingyun Yang, Suyi Li, Tianyu Feng +10
Text-to-image generation executes a diffusion workflow comprising multiple models centered on a base diffusion model. Existing serving systems treat each workflow as an opaque mono…
RTP-LLM: High-Performance Alibaba LLM Inference Engine
Boyu Tan, Jiarui Guo, Zongwei Lv +26
Large Language Models (LLMs) have revolutionized AI applications, but deploying them at scale presents significant challenges. We present RTP-LLM, a high-performance inference engi…
SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules
Suyi Li, Lingyun Yang, Xiaoxiao Jiang +12
Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud. A production T2I service typically involves a serving workflow where a b…
EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
Qingao Yi, Jiaang Duan, Hanwen Hu +10
Training large language models (LLMs) poses significant challenges regarding computational resources and memory capacity. Although distributed training techniques help mitigate the…
Joint Subcarrier and CPU Time Allocation for Mobile Edge Computing
Yinghao Yu, Jun Zhang, Khaled Ben Letaief
In mobile edge computing systems, mobile devices can offload compute-intensive tasks to a nearby cloudlet,so as to save energy and extend battery life. Unlike a fully-fledged cloud…
Inferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulation
Inferix Team, Tianyu Feng, Yizeng Han +13
World models serve as core simulators for fields such as agentic AI, embodied AI, and gaming, capable of generating long, physically realistic, and interactive high-quality videos.…
LERC: Coordinated Cache Management for Data-Parallel Systems
Yinghao Yu, Wei Wang, Jun Zhang +1
Memory caches are being aggressively used in today's data-parallel frameworks such as Spark, Tez and Storm. By caching input and intermediate data in memory, compute tasks can witn…
Dissecting Outlier Dynamics in LLM NVFP4 Pretraining
Peijie Dong, Ruibo Fan, Yuechen Tao +11
Training large language models using 4-bit arithmetic enhances throughput and memory efficiency. Yet, the limited dynamic range of FP4 increases sensitivity to outliers. While NVFP…
RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Training
Tianyuan Wu, Lunxi Cao, Yining Wei +11
Rollout-training disaggregation is emerging as the standard architecture for Reinforcement Learning (RL) post-training, where memory-bound rollout and compute-bound training are ph…
LRC: Dependency-Aware Cache Management for Data Analytics Clusters
Yinghao Yu, Wei Wang, Jun Zhang +1
Memory caches are being aggressively used in today's data-parallel systems such as Spark, Tez, and Piccolo. However, prevalent systems employ rather simple cache management policie…