Two-dimensional early exit optimisation of LLM inference

Jan Hůla, David Adamczyk, Tomáš Filip +2

We introduce a two-dimensional (2D) early exit strategy that coordinates layer-wise and sentence-wise exiting for classification tasks in large language models. By processing input…