139 citations · 141 across the 3 of their papers we have counts for
4 papers
OpWeave: Flexible Operator Disaggregation for Heterogeneous LLM Serving
Zikun Li, Yixuan Mei, Shiqi Pan +9
LLM serving systems increasingly disaggregate inference into finer-grained stages, with recent approaches separating attention from FFN or MoE execution during decode. This operato…
Demystifying and Improving Lazy Promotion in Cache Eviction
Qinghan Chen, Muhammad Haekal Muhyidin Al-Araby, Ziyue Qiu +3
Cache eviction algorithms play a critical role in the performance of modern data systems, yet their scalability is often limited by the high computational overhead associated with…
Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
Yixuan Mei, Yonghao Zhuang, Xupeng Miao +3
This paper introduces Helix, a distributed system for high-throughput, low-latency large language model (LLM) serving in heterogeneous GPU clusters. The key idea behind Helix is to…
DART: Dropouts meet Multiple Additive Regression Trees
K. V. Rashmi, Ran Gilad-Bachrach
Multiple Additive Regression Trees (MART), an ensemble model of boosted regression trees, is known to deliver high prediction accuracy for diverse tasks, and it is widely used in p…