48 citations · 62 across the 3 of their papers we have counts for
6 papers
Simple Recurrence Improves Masked Language Models
Tao Lei, Ran Tian, Jasmijn Bastings +1
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into t…
Scaling Up Models and Data with and
Adam Roberts, Hyung Won Chung, Anselm Levskaya +40
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…
The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?
Jasmijn Bastings, Katja Filippova
There is a recent surge of interest in using attention as explanation of model predictions, with mixed evidence on whether attention can be used as such. While attention convenient…
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
Ian Tenney, James Wexler, Jasmijn Bastings +8
We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models. We focus on core questions about model behavior: Why…
Interpretable Neural Predictions with Differentiable Binary Variables
Jasmijn Bastings, Wilker Aziz, Ivan Titov
The success of neural networks comes hand in hand with a desire for more interpretability. We focus on text classifiers and make them more interpretable by having them provide a ju…
Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks
Diego Marcheggiani, Jasmijn Bastings, Ivan Titov
Semantic representations have long been argued as potentially useful for enforcing meaning preservation and improving generalization performance of machine translation methods. In…