Fair Comparison between Efficient Attentions

Jiuk Hong, Chaehyeon Lee, Soyoun Bang +1

Transformers have been successfully used in various fields and are becoming the standard tools in computer vision. However, self-attention, a core component of transformers, has a…