76 citations · 116 across the 11 of their papers we have counts for
4 papers · 1 filter
How Language-Neutral is Multilingual BERT?
Jindřich Libovický, Rudolf Rosa, Alexander Fraser
Multilingual BERT (mBERT) provides sentence representations for 104 languages, which are useful for many multi-lingual tasks. Previous work probed the cross-linguality of mBERT usi…
Unsupervised Lemmatization as Embeddings-Based Word Clustering
Rudolf Rosa, Zdeněk Žabokrtský
We focus on the task of unsupervised lemmatization, i.e. grouping together inflected forms of one word under one label (a lemma) without the use of annotated training data. We prop…
Inducing Syntactic Trees from BERT Representations
Rudolf Rosa, David Mareček
We use the English model of BERT and explore how a deletion of one word in a sentence changes representations of other words. Our hypothesis is that removing a reducible word (e.g.…
From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions
David Mareček, Rudolf Rosa
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation. In many of the att…