AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving

Hao Mark Chen, Jinnan Guo, Wayne Luk +1

Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step. As decoding accelerates, tool e…