1 citations · 1 across the 7 of their papers we have counts for
4 papers · 1 filter
Semantic-Aware Scheduling for GPU Clusters with Large Language Models
Zerui Wang, Qinghao Hu, Ana Klimovic +4
Deep learning (DL) schedulers are pivotal in optimizing resource allocation in GPU clusters, but operate with a critical limitation: they are largely blind to the semantic context…
OctoPipe: Reducing Pipeline Bubbles for Heterogeneous Models via Co-Optimizing Partitioning, Placement, and Scheduling
Jihu Guo, Tenghui Ma, Wei Gao +6
Pipeline parallelism is widely used to train large language models (LLMs). However, increasing heterogeneity in model architectures exacerbates pipeline bubbles, thereby reducing t…
Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
Wei Gao, Xinyu Zhou, Peng Sun +2
Key-Value cache (\texttt{KV} \texttt{cache}) compression has emerged as a promising technique to optimize Large Language Model (LLM) serving. It primarily decreases the memory cons…
SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
Qiaoling Chen, Shenggui Li, Wei Gao +3
In recent years, Large Language Models (LLMs) have exhibited remarkable capabilities, driving advancements in real-world applications. However, training LLMs on increasingly long i…