Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
Energy-Aware Scheduling for Serverless LLM Serving on Shared GPUs
Tianyu Wang, Gourav Rattihalli, Aditya Dhakal +2
As LLM inference becomes a major cloud workload, its growing energy footprint makes cluster-wide energy optimization increasingly important. Serverless LLM serving helps platforms…
cs.DC2024
Improving GPU Multi-Tenancy Through Dynamic Multi-Instance GPU Reconfiguration
Tianyu Wang, Sheng Li, Bingyao Li +6
Continuous learning (CL) has emerged as one of the most popular deep learning paradigms deployed in modern cloud GPUs. Specifically, CL has the capability to continuously update th…
cs.DC2024
Improving Multi-Instance GPU Efficiency via Sub-Entry Sharing TLB Design
Bingyao Li, Yueqi Wang, Tianyu Wang +4
NVIDIA's Multi-Instance GPU (MIG) technology enables partitioning GPU computing power and memory into separate hardware instances, providing complete isolation including compute re…