Staying True to Your Word: (How) Can Attention Become Explanation?

Martin Tutek, Jan Šnajder

The attention mechanism has quickly become ubiquitous in NLP. In addition to improving performance of models, attention has been widely used as a glimpse into the inner workings of…