LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

SangLyul Cho, Langqing Cui, Sehoon Kim +2

Looped Transformers achieve strong performance with compact parameter sizes by repeatedly applying a shared stack of Transformer blocks across recurrent depths. However, they incur…