Speculate with Memory: Lossless Acceleration for LLM Agents

Yu Li, Qinyuan Ye, Prafulla Kumar Choubey +2

Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle. However, existing speculators…