Semantic Tagging with Deep Residual Networks
arXiv:1609.07053
Abstract
We propose a novel semantic tagging task, sem-tagging, tailored for the purpose of multilingual semantic parsing, and present the first tagger using deep residual networks (ResNets). Our tagger uses both word and character representations and includes a novel residual bypass architecture. We evaluate the tagset both intrinsically on the new task of semantic tagging, as well as on Part-of-Speech (POS) tagging. Our system, consisting of a ResNet and an auxiliary loss function predicting our semantic tags, significantly outperforms prior results on English Universal Dependencies POS tagging (95.71% accuracy on UD v1.2 and 95.67% accuracy on UD v1.3).
COLING 2016, camera ready version
References in corpus (8)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Deep Residual Learning for Image Recognition
- Multi-Task Cross-Lingual Sequence Tagging from Scratch
- A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding
- Text segmentation with character-level text embeddings
- Byte-based Language Identification with Deep Convolutional Networks
Cited by in corpus (14)
- Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks
- Multi-Task Learning for Sequence Tagging: An Empirical Study
- Towards Universal Semantic Tagging
- The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations
- Exploring and Predicting Transferability across NLP Tasks
- Emergence of Separable Manifolds in Deep Language Representations
- Parsing Thai Social Data: A New Challenge for Thai NLP
- Shallow Syntax in Deep Water
- Byte-based Language Identification with Deep Convolutional Networks
- The Parallel Meaning Bank: A Framework for Semantically Annotating Multiple Languages
- Exploring Neural Methods for Parsing Discourse Representation Structures
- Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT
- Multitask and Multilingual Modelling for Lexical Analysis
- Exploiting a Zoo of Checkpoints for Unseen Tasks