4 papers
TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
Zhuohang Bian, Feiyang Wu, Zhuoran Li +2
Large Language Models (LLMs) are increasingly deployed in complex multi-agent applications that rely on external function calls. This workload creates severe performance challenges…
TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
Zhuohang Bian, Feiyang Wu, Chengrui Zhang +3
Multi-agent LLM applications organize execution in synchronized rounds where a central scheduler gathers outputs from all agents and redistributes the combined context. This All-Ga…
Mitigating Ambiguities in 3D Classification with Gaussian Splatting
Ruiqi Zhang, Hao Zhu, Jingyi Zhao +3
3D classification with point cloud input is a fundamental problem in 3D vision. However, due to the discrete nature and the insufficient material description of point cloud represe…
vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving
Jiale Xu, Rui Zhang, Cong Guo +11
Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. This surge in demand poses significant challenges in optimizing throughp…