2 papers
cs.DC2026
Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving
Yifan Sui, Han Zhao, Rui Ma +6
LLM-powered agents execute tasks through a sequential loop of model generation and tool execution. Today's serving systems serialize this loop, leaving tool latency exposed on the…
cs.LG2025
ServerlessLoRA: Minimizing Latency and Cost in Serverless Inference for LoRA-Based LLMs
Yifan Sui, Hao Wang, Hanfei Yu +2
Serverless computing has grown rapidly for serving Large Language Model (LLM) inference due to its pay-as-you-go pricing, fine-grained GPU usage, and rapid scaling. However, our an…