A Survey on Neural Network Language Models
arXiv:1906.03591
Abstract
As the core component of Natural Language Processing (NLP) system, Language Model (LM) can provide word representation and probability indication of word sequences. Neural Network Language Models (NNLMs) overcome the curse of dimensionality and improve the performance of traditional LMs. A survey on NNLMs is performed in this paper. The structure of classic NNLMs is described firstly, and then some major improvements are introduced and analyzed. We summarize and compare corpora and toolkits of NNLMs. Further, some research directions of NNLMs are discussed.
References in corpus (9)
- Efficient Estimation of Word Representations in Vector Space
- A Neural Knowledge Language Model
- Improving Neural Language Models with a Continuous Cache
- Syntactic and Semantic Features For Code-Switching Factored Language Models
- Larger-Context Language Modelling
- Gated Word-Character Recurrent Language Model
- TheanoLM - An Extensible Toolkit for Neural Network Language Modeling
- Character-Word LSTM Language Models
- Character-Level Language Modeling with Hierarchical Recurrent Neural Networks