Accelerating Attention through Gradient-Based Learned Runtime Pruning

Zheng Li, Soroush Ghodrati, Amir Yazdanbakhsh +2

Self-attention is a key enabler of state-of-art accuracy for various transformer-based Natural Language Processing models. This attention mechanism calculates a correlation score f…