RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inference

Lianming Huang, Shangyu Wu, Yufei Cui +6

Deploying large language model inference remains challenging due to their high computational overhead. Early exit optimizes model inference by adaptively reducing the number of inf…