Deep Semi-Supervised Learning with Linguistically Motivated Sequence Labeling Task Hierarchies

Abstract

In this paper we present a novel Neural Network algorithm for conducting semi-supervised learning for sequence labeling tasks arranged in a linguistically motivated hierarchy. This relationship is exploited to regularise the representations of supervised tasks by backpropagating the error of the unsupervised task through the supervised tasks. We introduce a neural network where lower layers are supervised by junior downstream tasks and the final layer task is an auxiliary unsupervised task. The architecture shows improvements of up to two percentage points F1 for Chunking compared to a plausible baseline.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Introduction to the CoNLL-2000 Shared Task: Chunking
• Multi-Task Cross-Lingual Sequence Tagging from Scratch
• Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

Cited by in corpus (2)

• Multi-Task Learning for Sequence Tagging: An Empirical Study
• On mechanisms for transfer using landmark value functions in multi-task lifelong reinforcement learning