FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

Shuxuan Li, Jialiang Kang, Yanghao Zhou +2

Speculative decoding accelerates LLM generation through lightweight drafting and parallel verification. It is particularly promising for long inputs, where KV-cache access costs ca…