Attention-based Memory Selection Recurrent Network for Language Modeling

Da-Rong Liu, Shun-Po Chuang, Hung-yi Lee

Recurrent neural networks (RNNs) have achieved great success in language modeling. However, since the RNNs have fixed size of memory, their memory cannot store all the information…