Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving

Yifan Sui, Han Zhao, Rui Ma +6

LLM-powered agents execute tasks through a sequential loop of model generation and tool execution. Today's serving systems serialize this loop, leaving tool latency exposed on the…