13 citations · 13 across the 2 of their papers we have counts for
3 papers
cs.CL2020
DoLFIn: Distributions over Latent Features for Interpretability
Phong Le, Willem Zuidema
Interpreting the inner workings of neural models is a key step in ensuring the robustness and trustworthiness of the models, but work on neural network interpretability typically f…
cs.LG2020★ 13 cited
Transferring Inductive Biases through Knowledge Distillation
Samira Abnar, Mostafa Dehghani, Willem Zuidema
Having the right inductive biases can be crucial in many tasks or scenarios where data or computing resources are a limiting factor, or where training data is not perfectly represe…
cs.LG2020
Quantifying Attention Flow in Transformers
Samira Abnar, Willem Zuidema
In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of th…