OoO-Spec: Out-of-Order Semantic Speculation for Fast Tool Calling

Zhiheng Zhang, Mujie Xu, Feiyu Sun +1

LLMs generate tool calls token by token, even though the function choice and argument values can often be predicted in parallel from the request and tool schema. ToolSpec reduces t…