13 citations · 13 across the 1 of their papers we have counts for
3 papers
cs.CL2019★ 13 cited
Inducing Syntactic Trees from BERT Representations
Rudolf Rosa, David Mareček
We use the English model of BERT and explore how a deletion of one word in a sentence changes representations of other words. Our hypothesis is that removing a reducible word (e.g.…
cs.CL2019
Derivational Morphological Relations in Word Embeddings
Tomáš Musil, Jonáš Vidra, David Mareček
Derivation is a type of a word-formation process which creates new words from existing ones by adding, changing or deleting affixes. In this paper, we explore the potential of word…
cs.CL2019
From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions
David Mareček, Rudolf Rosa
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation. In many of the att…