1 citations · 2 across the 3 of their papers we have counts for
9 papers
Word Order Does Matter (And Shuffled Language Models Know It)
Vinit Ravishankar, Mostafa Abdou, Artur Kulmizev +1
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the import…
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich +3
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). Howe…
Attention Can Reflect Syntactic Structure (If You Let It)
Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou +2
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head atten…
Positional Artefacts Propagate Through Masked Language Model Embeddings
Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across la…
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev +2
We present Køpsala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for…
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur Kulmizev, Vinit Ravishankar, Mostafa Abdou +1
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces. Howev…