Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning

Yanyu Zhu, Hoilam Pao, Niu Hu +6

Large Language Models suffer from slow autoregressive inference. While self-speculative decoding accelerates this process, its efficiency is hampered by static configurations like…