14 citations · 27 across the 4 of their papers we have counts for
5 papers
Transferring Inductive Biases through Knowledge Distillation
Samira Abnar, Mostafa Dehghani, Willem Zuidema
Having the right inductive biases can be crucial in many tasks or scenarios where data or computing resources are a limiting factor, or where training data is not perfectly represe…
Quantifying Attention Flow in Transformers
Samira Abnar, Willem Zuidema
In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of th…
A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings
Niels van der Heijden, Samira Abnar, Ekaterina Shutova
The lack of annotated data in many languages is a well-known challenge within the field of multilingual natural language processing (NLP). Therefore, many recent studies focus on z…
Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains
Samira Abnar, Lisa Beinborn, Rochelle Choenni +1
In this paper, we define and apply representational stability analysis (ReStA), an intuitive way of analyzing neural language models. ReStA is a variant of the popular representati…
Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity
Samira Abnar, Rasyan Ahmed, Max Mijnheer +1
We evaluate 8 different word embedding models on their usefulness for predicting the neural activation patterns associated with concrete nouns. The models we consider include an ex…