Language Modeling With Factorization Memory

Lee Xiong, Maksim Tkachenko, Johanes Effendi +1

We propose Factorization Memory, an efficient recurrent neural network (RNN) architecture that achieves performance comparable to Transformer models on short-context language model…