34 citations · 38 across the 7 of their papers we have counts for
Showing 2021Show all
2 papers · 1 filter
cs.CL2021
The Impact of Positional Encodings on Multilingual Compression
Vinit Ravishankar, Anders Søgaard
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional enc…
cs.CL2021
Attention Can Reflect Syntactic Structure (If You Let It)
Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou +2
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head atten…