10 citations · 10 across the 6 of their papers we have counts for
6 papers
SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving
Pol G. Recasens, Ferran Agullo, Yue Zhu +3
Large language model (LLM) serving commonly increases batch size to improve throughput, but performance eventually reaches a deployment-dependent plateau beyond which larger batche…
Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems
Yuzhe Zhang, Feiran Liu, Yi Shan +8
Large language models are increasingly deployed in multi-agent systems to overcome context limitations by distributing information across agents. Yet whether agents can reliably co…
Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
Jiaxi Li, Yue Zhu, Eun Kyung Lee +1
Different from traditional Large Language Model (LLM) serving that colocates the prefill and decode stages on the same GPU, disaggregated serving dedicates distinct GPUs to prefill…
Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
Yue Zhu, Hao Yu, Chen Wang +2
The increasing adoption of large language models (LLMs) with extended context windows necessitates efficient Key-Value Cache (KVC) management to optimize inference performance. Inf…
Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
Pol G. Recasens, Ferran Agullo, Yue Zhu +5
Large language models have been widely adopted across different tasks, but their auto-regressive generation nature often leads to inefficient resource utilization during inference.…
Towards Pareto Optimal Throughput in Small Language Model Serving
Pol G. Recasens, Yue Zhu, Chen Wang +5
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory deman…