Saliency-driven Dynamic Token Pruning for Large Language Models

Yao Tao, Yehui Tang, Yun Wang +3

Despite the recent success of large language models (LLMs), LLMs are particularly challenging in long-sequence inference scenarios due to the quadratic computational complexity of…