Deep Contextualized Self-training for Low Resource Dependency Parsing
arXiv:1911.04286
Abstract
Neural dependency parsing has proven very effective, achieving state-of-the-art results on numerous domains and languages. Unfortunately, it requires large amounts of labeled data, that is costly and laborious to create. In this paper we propose a self-training algorithm that alleviates this annotation bottleneck by training a parser on its own output. Our Deep Contextualized Self-training (DCST) algorithm utilizes representation models trained on sequence labeling tasks that are derived from the parser's output when applied to unlabeled data, and integrates these models with the base parser through a gating mechanism. We conduct experiments across multiple languages, both in low resource in-domain and in cross-domain setups, and demonstrate that DCST substantially outperforms traditional self-training as well as recent semi-supervised training methods.
Accepted to TACL in September 2019
References in corpus (5)
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
- A Transition-Based Directed Acyclic Graph Parser for UCCA
- No Training Required: Exploring Random Encoders for Sentence Classification
- A Dependency-Based Neural Reordering Model for Statistical Machine Translation