Not All Attention Is Needed: Gated Attention Network for Sequence Data

Lanqing Xue, Xiaopeng Li, Nevin L. Zhang

Although deep neural networks generally have fixed network structures, the concept of dynamic mechanism has drawn more and more attention in recent years. Attention mechanisms comp…