Goose: Anisotropic Speculation Trees for Training-Free Speculative Decoding

Tao Jin, Phuong Minh Nguyen, Naoya Inoue

Speculative decoding accelerates large language model inference by drafting multiple candidate tokens and verifying them in a single forward pass. Candidates are organized as a tre…