The Diminishing Returns of Early-Exit Decoding in Modern LLMs

Rui Wei, Rui Du, Hanfei Yu +4

In Large Language Model (LLM) inference, early-exit refers to stopping computation at an intermediate layer once the prediction is sufficiently confident, thereby reducing latency…