READER: Retrieval-Assisted Drafter for Efficient LLM Inference

Maxim Divilkovskiy, Vitaly Malygin, Sergey Zlobin +6

Autoregressive Language Models instantiate a factorized likelihood over token sequences, yet their strictly sequential decoding process imposes an intrinsic lower bound on inferenc…