Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs

Abstract

We present extensions to a continuous-state dependency parsing method that makes it applicable to morphologically rich languages. Starting with a high-performance transition-based parser that uses long short-term memory (LSTM) recurrent neural networks to learn representations of the parser state, we replace lookup-based word representations with representations constructed from the orthographic representations of the words, also using LSTMs. This allows statistical sharing across word forms that are similar on the surface. Experiments for morphologically rich languages show that the parsing model benefits from incorporating the character-based encodings of words.

In Proceedings of EMNLP 2015

References in corpus (3)

• LSTM: A Search Space Odyssey
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Structured Training for Neural Network Transition-Based Parsing

Cited by in corpus (39)

• Exploring the Limits of Language Modeling
• Recent Advances in Recurrent Neural Networks
• Enriching Word Vectors with Subword Information
• Character-based Neural Machine Translation
• Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
• Attending to Characters in Neural Sequence Labeling Models
• Charagram: Embedding Words and Sentences via Character n-grams
• Neural Probabilistic Model for Non-projective MST Parsing
• SyntaxNet Models for the CoNLL 2017 Shared Task
• Multilingual Language Processing From Bytes
• Character-Word LSTM Language Models
• Neural Morphological Tagging from Characters for Morphologically Rich Languages
• Exploiting Multi-typed Treebanks for Parsing with Deep Multi-task Learning
• Character Composition Model with Convolutional Neural Networks for Dependency Parsing on Morphologically Rich Languages
• Bi-directional Attention with Agreement for Dependency Parsing
• Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network
• Scientific Information Extraction with Semi-supervised Neural Tagging
• Keystroke dynamics as signal for shallow syntactic parsing
• Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
• Character-based NMT with Transformer
• Review Helpfulness Prediction with Embedding-Gated CNN
• Parsing Tweets into Universal Dependencies
• Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing
• A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding
• What Taggers Fail to Learn, Parsers Need the Most
• Joint POS Tagging and Dependency Parsing with Transition-based Neural Networks
• Shift-Reduce Constituent Parsing with Neural Lookahead Features
• Stack-propagation: Improved Representation Learning for Syntax
• Information Extraction from Scientific Literature for Method Recommendation
• Transition-Based Dependency Parsing using Perceptron Learner
• What do character-level models learn about morphology? The case of dependency parsing
• An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing
• Joint Semantic Synthesis and Morphological Analysis of the Derived Word
• Encoder-Decoder Shift-Reduce Syntactic Parsing
• Authorship Attribution in Bangla literature using Character-level CNN
• Fill it up: Exploiting partial dependency annotations in a minimum spanning tree parser
• Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval
• Multi-task Learning for Low-resource Second Language Acquisition Modeling
• On Multilingual Training of Neural Dependency Parsers