HARP: Hesitation-Aware Reframing in Transformer Inference Pass

Romain Storaï, Seung-won Hwang

This paper aims to improve the performance of large language models by addressing the variable computational demands in inference steps, where some tokens require more computationa…