1 citations · 3 across the 5 of their papers we have counts for
9 papers
Word Order Does Matter (And Shuffled Language Models Know It)
Vinit Ravishankar, Mostafa Abdou, Artur Kulmizev +1
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the import…
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich +3
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). Howe…
Attention Can Reflect Syntactic Structure (If You Let It)
Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou +2
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head atten…
Positional Artefacts Propagate Through Masked Language Model Embeddings
Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across la…
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev +2
We present Køpsala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for…
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur Kulmizev, Vinit Ravishankar, Mostafa Abdou +1
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces. Howev…
Higher-order Comparisons of Sentence Encoder Representations
Mostafa Abdou, Artur Kulmizev, Felix Hill +2
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e.g., fMRI, electrophysi…
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisited
Artur Kulmizev, Miryam de Lhoneux, Johannes Gontrum +2
Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural feat…
What can we learn from Semantic Tagging?
Mostafa Abdou, Artur Kulmizev, Vinit Ravishankar +2
We investigate the effects of multi-task learning using the recently introduced task of semantic tagging. We employ semantic tagging as an auxiliary task for three different NLP ta…