2 citations · 2 across the 3 of their papers we have counts for
4 papers
Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving
Jiantong Jiang, Yue Yang, Peiyu Yang +1
Large language model (LLM) serving is increasingly constrained by the GPU memory consumed by key-value (KV) caches. Existing compression, eviction, and offloading techniques allevi…
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Jiantong Jiang, Peiyu Yang, Rui Zhang +1
Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during aut…
Coordinated Scheduling for MoE LLM Serving
Yifan Sun, Zhexiang Zhang, Jiantong Jiang +5
Serving Mixture-of-Experts (MoE) large language models (LLMs) is challenging because dynamic request workloads interact with sparse expert routing, creating both data-parallel (DP)…
CleanCodec: Efficient and Robust Speech Tokenization via Perceptually Guided Encoding
Eugene Kwek, Feng Liu, Rui Zhang +1
Neural audio codecs are a key component of speech processing pipelines, compressing audio into discrete tokens for downstream modeling. However, existing codecs struggle to balance…