Revisiting Attention Weights as Explanations from an Information Theoretic Perspective

Bingyang Wen, K. P. Subbalakshmi, Fan Yang

Attention mechanisms have recently demonstrated impressive performance on a range of NLP tasks, and attention scores are often used as a proxy for model explainability. However, th…