11 citations · 14 across the 14 of their papers we have counts for
15 papers
From Overloaded to Guaranteed: High-Throughput Multi-SLO Enforcement for LoRA-Assisted On-Premise LLM Deployment
Zeshen Zhang, Han Zhao, Weihao Cui +9
As Large Language Models (LLMs) become essential in privacy-sensitive sectors like hospitals and government agencies, the on-premise LLM servers offer a cost-effective and secure a…
Dissecting How Die Scaling Breaks GPU Fine-grained Scheduling
Xiaoze Fan, Jianhao Wang, Weihao Cui +8
Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific com…
MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters via Spatial-Temporal Backbone Multiplexing
Chunyu Xue, Yi Pan, Weihao Cui +4
Parameter-Efficient Fine-Tuning (PEFT) is widely applied as the backend of fine-tuning APIs for large language model (LLM) customization in datacenters. Service providers deploy se…
Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
Ao Xu, Han Zhao, Weihao Cui +7
Large language models (LLMs) are increasingly deployed under the Model-as-a-Service (MaaS) paradigm. To meet stringent quality-of-service (QoS) requirements, existing LLM serving s…
Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution
Shulai Zhang, Ao Xu, Quan Chen +6
Embodied AI systems operate in dynamic environments, requiring seamless integration of perception and generation modules to process high-frequency input and output demands. Traditi…
Towards High-Goodput LLM Serving with Prefill-decode Multiplexing
Yukang Chen, Weihao Cui, Han Zhao +7
Large Language Model (LLM) serving must meet stringent Service Level Objectives (SLOs) for both the prefill and decode phases. Some existing solutions disaggregate the two phases,…
Efficient Function-as-a-Service for Large Language Models with TIDAL
Weihao Cui, Ziyi Xu, Han Zhao +4
Large Language Model (LLM) applications have emerged as a prominent use case for Function-as-a-Service (FaaS) due to their high computational demands and sporadic invocation patter…
Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
Shulai Zhang, Ningxin Zheng, Haibin Lin +9
Mixture-of-experts (MoE) has been extensively employed to scale large language models to trillion-plus parameters while maintaining a fixed computational cost. The development of l…
VQ-LLM: High-performance Code Generation for Vector Quantization Augmented LLM Inference
Zihan Liu, Xinhao Luo, Junxian Guo +11
In this work, we design and implement VQ-LLM, an efficient fused Vector Quantization (VQ) kernel generation framework. We first introduce a software abstraction called codebook cac…
Vortex: Efficient Sample-Free Dynamic Tensor Program Optimization via Hardware-aware Strategy Space Hierarchization
Yangjie Zhou, Honglin Zhu, Qian Qiu +9
Dynamic-shape deep neural networks (DNNs) are rapidly evolving, attracting attention for their ability to handle variable input sizes in real-time applications. However, existing c…
The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving
Pai Zeng, Zhenyu Ning, Jieru Zhao +5
We survey the large language model (LLM) serving area to understand the intricate dynamics between cost-efficiency and accuracy, which is magnified by the growing need for longer c…
Towards Fast Setup and High Throughput of GPU Serverless Computing
Han Zhao, Weihao Cui, Quan Chen +6
Integrating GPUs into serverless computing platforms is crucial for improving efficiency. However, existing solutions for GPU-enabled serverless computing platforms face two signif…
Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
Chunyu Xue, Weihao Cui, Quan Chen +10
Efficiently training large-scale models (LMs) in GPU clusters involves two separate avenues: inter-job dynamic scheduling and intra-job adaptive parallelism (AP). However, existing…
Accelerating Sparse DNNs Based on Tiled GEMM
Cong Guo, Fengchen Xue, Jingwen Leng +5
Network pruning can reduce the computation cost of deep neural network (DNN) models. However, sparse models often produce randomly-distributed weights to maintain accuracy, leading…
AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
Yangjie Zhou, Yaoxu Song, Jingwen Leng +7
Graph neural networks (GNNs) are powerful tools for exploring and learning from graph structures and features. As such, achieving high-performance execution for GNNs becomes crucia…