Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models

Joseph F DeRose, Jiayao Wang, Matthew Berger

Advances in language modeling have led to the development of deep attention-based models that are performant across a wide variety of natural language processing (NLP) problems. Th…