7 citations · 9 across the 6 of their papers we have counts for
4 papers · 1 filter
FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution
Shuo Yang, Xiaoze Fan, Melissa Pan +8
Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system th…
Flash-KMeans: Fast and Memory-Efficient Exact K-Means
Shuo Yang, Haocheng Xi, Yilong Zhao +10
-means has historically been positioned primarily as an offline processing primitive, typically used for dataset organization or embedding preprocessing rather than as a first-c…
Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning
Shan Yu, Yifan Qiao, Mingyuan Ma +18
Inference providers must maintain availability for many LLMs, including low-volume but essential models, making resource efficiency increasingly important as token prices fall. Ana…
ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
Yifan Qiao, Shu Anzai, Shan Yu +10
Large language model (LLM) serving demands low latency and high throughput, but high load variability makes it challenging to achieve high GPU utilization. In this paper, we identi…