101 citations · 261 across the 12 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
cs.CL2019
Analyzing the Structure of Attention in a Transformer Language Model
Jesse Vig, Yonatan Belinkov
The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks. In this paper, we analyze the s…
cs.HC2019
A Multiscale Visualization of Attention in the Transformer Model
Jesse Vig
The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of us…
cs.HC2019★ 29 cited
Visualizing Attention in Transformer-Based Language Representation Models
Jesse Vig
We present an open-source tool for visualizing multi-head self-attention in Transformer-based language representation models. The tool extends earlier work by visualizing attention…