RASD: Retrieval-Augmented Speculative Decoding

Guofeng Quan, Wenfeng Feng, Chuzhan Hao +3

Speculative decoding accelerates inference in large language models (LLMs) by generating draft tokens for target model verification. Current approaches for obtaining draft tokens r…