13 citations · 23 across the 3 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2020★ 13 cited
Transferring Inductive Biases through Knowledge Distillation
Samira Abnar, Mostafa Dehghani, Willem Zuidema
Having the right inductive biases can be crucial in many tasks or scenarios where data or computing resources are a limiting factor, or where training data is not perfectly represe…
cs.LG2020
Quantifying Attention Flow in Transformers
Samira Abnar, Willem Zuidema
In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of th…