BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration

Chence Yang, Ningxi Cheng, Arash Akbari +9

Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, existing methods often requir…