Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification

Guang Huang, Zeyi Wen

Speculative Decoding (SD) has emerged as a premier technique for accelerating Large Language Model (LLM) inference by decoupling token generation into rapid drafting and parallel v…