Exploring Attention Map Reuse for Efficient Transformer Neural Networks

Kyuhong Shim, Jungwook Choi, Wonyong Sung

Transformer-based deep neural networks have achieved great success in various sequence applications due to their powerful ability to model long-range dependency. The key module of…