Rényi Attention Entropy for Patch Pruning

Hiroaki Aizawa, Yuki Igaue

Transformers are strong baselines in both vision and language because self-attention captures long-range dependencies across tokens. However, the cost of self-attention grows quadr…