Is Sparse Attention more Interpretable?

Clara Meister, Stefan Lazov, Isabelle Augenstein +1

Sparse attention has been claimed to increase model interpretability under the assumption that it highlights influential inputs. Yet the attention distribution is typically over re…