1 citations · 1 across the 8 of their papers we have counts for
8 papers
Dissecting How Die Scaling Breaks GPU Fine-grained Scheduling
Xiaoze Fan, Jianhao Wang, Weihao Cui +8
Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific com…
Approaching Shannon Bound with Lossless LLM Weight Compression
Hongshi Tan, Yao Chen, Gustavo Alonso +2
Large language models (LLMs) now scale to trillions of parameters, driving weight storage into the terabyte regime and creating an acute mismatch with GPU memory capacity. Although…
MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters via Spatial-Temporal Backbone Multiplexing
Chunyu Xue, Yi Pan, Weihao Cui +4
Parameter-Efficient Fine-Tuning (PEFT) is widely applied as the backend of fine-tuning APIs for large language model (LLM) customization in datacenters. Service providers deploy se…
Towards High-Goodput LLM Serving with Prefill-decode Multiplexing
Yukang Chen, Weihao Cui, Han Zhao +7
Large Language Model (LLM) serving must meet stringent Service Level Objectives (SLOs) for both the prefill and decode phases. Some existing solutions disaggregate the two phases,…
Efficient Function-as-a-Service for Large Language Models with TIDAL
Weihao Cui, Ziyi Xu, Han Zhao +4
Large Language Model (LLM) applications have emerged as a prominent use case for Function-as-a-Service (FaaS) due to their high computational demands and sporadic invocation patter…
Flare: Anomaly Diagnostics for Divergent LLM Training in GPU Clusters of Thousand-Plus Scale
Weihao Cui, Ji Zhang, Han Zhao +5
The rapid proliferation of large language models has driven the need for efficient GPU training clusters. However, it is challenging due to the frequent occurrence of training anom…
Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
Chunyu Xue, Weihao Cui, Quan Chen +10
Efficiently training large-scale models (LMs) in GPU clusters involves two separate avenues: inter-job dynamic scheduling and intra-job adaptive parallelism (AP). However, existing…
Accelerating Generative Neural Networks on Unmodified Deep Learning Processors -- A Software Approach
Dawen Xu, Ying Wang, Kaijie Tu +3
Generative neural network is a new category of neural networks and it has been widely utilized in applications such as content generation, unsupervised learning, segmentation and p…