331 citations · 478 across the 17 of their papers we have counts for
22 papers
Schrödinger's Tree -- On Syntax and Neural Language Models
Artur Kulmizev, Joakim Nivre
In the last half-decade, the field of natural language processing (NLP) has undergone two major transitions: the switch to neural networks as the primary modeling paradigm and the…
Revisiting Negation in Neural Machine Translation
Gongbo Tang, Philipp Rönchen, Rico Sennrich +1
In this paper, we evaluate the translation of negation both automatically and manually, in English--German (EN--DE) and English--Chinese (EN--ZH). We show that the ability of neura…
Syntactic Nuclei in Dependency Parsing -- A Multilingual Exploration
Ali Basirat, Joakim Nivre
Standard models for syntactic dependency parsing take words to be the elementary units that enter into dependency relations. In this paper, we investigate whether there are any ben…
Attention Can Reflect Syntactic Structure (If You Let It)
Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou +2
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head atten…
Greedy Transition-Based Dependency Parsing with Discrete and Continuous Supertag Features
Ali Basirat, Joakim Nivre
We study the effect of rich supertag features in greedy transition-based dependency parsing. While previous studies have shown that sparse boolean features representing the 1-best…
Principal Word Vectors
Ali Basirat, Christian Hardmeier, Joakim Nivre
We generalize principal component analysis for embedding words into a vector space. The generalization is made in two major levels. The first is to generalize the concept of the co…
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev +2
We present Køpsala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for…
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur Kulmizev, Vinit Ravishankar, Mostafa Abdou +1
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces. Howev…
Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter +6
Universal Dependencies is an open community effort to create cross-linguistically consistent treebank annotation for many languages within a dependency-based lexicalist framework.…
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing -- A Tale of Two Parsers Revisited
Artur Kulmizev, Miryam de Lhoneux, Johannes Gontrum +2
Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural feat…
Encoders Help You Disambiguate Word Senses in Neural Machine Translation
Gongbo Tang, Rico Sennrich, Joakim Nivre
Neural machine translation (NMT) has achieved new state-of-the-art performance in translating ambiguous words. However, it is still unclear which component dominates the process of…
What Should/Do/Can LSTMs Learn When Parsing Auxiliary Verb Constructions?
Miryam de Lhoneux, Sara Stymne, Joakim Nivre
There is a growing interest in investigating what neural NLP models learn about language. A prominent open question is the question of whether or not it is necessary to model hiera…
Recursive Subtree Composition in LSTM-Based Dependency Parsing
Miryam de Lhoneux, Miguel Ballesteros, Joakim Nivre
The need for tree structure modelling on top of sequence modelling is an open issue in neural dependency parsing. We investigate the impact of adding a tree layer on top of a seque…
An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation
Gongbo Tang, Rico Sennrich, Joakim Nivre
Recent work has shown that the encoder-decoder attention mechanisms in neural machine translation (NMT) are different from the word alignment in statistical machine translation. In…
82 Treebanks, 34 Models: Universal Dependency Parsing with Multi-Treebank Models
Aaron Smith, Bernd Bohnet, Miryam de Lhoneux +3
We present the Uppsala system for the CoNLL 2018 Shared Task on universal dependency parsing. Our system is a pipeline consisting of three components: the first performs joint word…
An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing
Aaron Smith, Miryam de Lhoneux, Sara Stymne +1
We provide a comprehensive analysis of the interactions between pre-trained word embeddings, character models and POS tags in a transition-based dependency parser. While previous s…
Universal Word Segmentation: Implementation and Interpretation
Yan Shao, Christian Hardmeier, Joakim Nivre
Word segmentation is a low-level NLP task that is non-trivial for a considerable number of languages. In this paper, we present a sequence tagging framework and apply it to word se…
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization
Gongbo Tang, Fabienne Cap, Eva Pettersson +1
In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages: English, German, Hungarian, Icelandic, and Swedish. The NMT mod…
Parser Training with Heterogeneous Treebanks
Sara Stymne, Miryam de Lhoneux, Aaron Smith +1
How to make the most of multiple heterogeneous treebanks when training a monolingual dependency parser is an open question. We start by investigating previously suggested, but litt…
Sentences with Gapping: Parsing and Reconstructing Elided Predicates
Sebastian Schuster, Joakim Nivre, Christopher D. Manning
Sentences with gapping, such as Paul likes coffee and Mary tea, lack an overt predicate to indicate the relation between two or more arguments. Surface syntax representations of su…
Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF
Yan Shao, Christian Hardmeier, Jörg Tiedemann +1
We present a character-based model for joint segmentation and POS tagging for Chinese. The bidirectional RNN-CRF architecture for general sequence tagging is adapted and applied wi…
Static and Dynamic Feature Selection in Morphosyntactic Analyzers
Bernd Bohnet, Miguel Ballesteros, Ryan McDonald +1
We study the use of greedy feature selection methods for morphosyntactic tagging under a number of different conditions. We compare a static ordering of features to a dynamic order…