Attending to Characters in Neural Sequence Labeling Models
arXiv:1611.04361
Abstract
Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words. We investigate character-level extensions to such models and propose a novel architecture for combining alternative word representations. By using an attention mechanism, the model is able to dynamically decide how much information to use from a word- or character-level component. We evaluated different architectures on a range of sequence labeling datasets, and character-level extensions were found to improve performance on every benchmark. In addition, the proposed attention-based architecture delivered the best results even with a smaller number of trainable parameters.
Proceedings of COLING 2016
References in corpus (8)
- ADADELTA: An Adaptive Learning Rate Method
- Natural Language Processing (almost) from Scratch
- Character-Aware Neural Language Models
- Introduction to the CoNLL-2000 Shared Task: Chunking
- Character-based Neural Machine Translation
- Compositional Sequence Labeling Models for Error Detection in Learner Writing
- Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
- Gated Word-Character Recurrent Language Model
Cited by in corpus (9)
- Graph Convolutional Networks for Named Entity Recognition
- Probing Biomedical Embeddings from Language Models
- Automatic Identification of Indicators of Compromise using Neural-Based Sequence Labelling
- A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
- Neural Multi-task Learning in Automated Assessment
- Named Entity Recognition in the Legal Domain using a Pointer Generator Network
- GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition
- Multi-Task Learning for Argumentation Mining
- Character-based Neural Networks for Sentence Pair Modeling