Character-Word LSTM Language Models
arXiv:1704.02813
Abstract
We present a Character-Word Long Short-Term Memory Language Model which both reduces the perplexity with respect to a baseline word-level language model and reduces the number of parameters of the model. Character information can reveal structural (dis)similarities between words and can even be used when a word is out-of-vocabulary, thus improving the modeling of infrequent and unknown words. By concatenating word and character embeddings, we achieve up to 2.77% relative improvement on English compared to a baseline model with a similar amount of parameters and 4.57% on Dutch. Moreover, we also outperform baseline word-level models with a larger number of parameters.
References in corpus (9)
- Recurrent Neural Network Regularization
- Training Very Deep Networks
- Character-Aware Neural Language Models
- Exploring the Limits of Language Modeling
- Character-based Neural Machine Translation
- Neural Language Correction with Character-Based Attention
- Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
- Gated Word-Character Recurrent Language Model
- Alternative structures for character-level RNNs
Cited by in corpus (6)
- A Survey on Neural Network Language Models
- Subword-augmented Embedding for Cloze Reading Comprehension
- BioNerFlair: biomedical named entity recognition using flair embedding and sequence tagger
- Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
- Joint Embedding Learning of Educational Knowledge Graphs
- Patterns versus Characters in Subword-aware Neural Language Modeling