Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design

Zixiao Huang, Wen Zeng, Tianyu Fu +10

LLM-based search agents achieve strong performance but suffer from severe latency, as each step requires serialized LLM reasoning followed by action of tool execution. We revisit t…