Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS
Omar Basit, Yunzhao Liu, Z. Jonny Kong +1
Prefill/decode disaggregation is increasingly adopted in LLM serving to improve the latency-throughput tradeoff and meet strict TTFT and TPOT SLOs. However, LLM inference remains e…
cs.DC2025
Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
Yunzhao Liu, Qiang Xu, Y. Charlie Hu
Efficient LLM inference is critical for real-world applications, especially within heterogeneous GPU clusters commonly found in organizations and on-premise datacenters as GPU arch…
cs.DC2025
PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
Z. Jonny Kong, Qiang Xu, Y. Charlie Hu
With the rapid innovation of GPUs, heterogeneous GPU clusters in both public clouds and on-premise data centers have become increasingly commonplace. In this paper, we demonstrate…