RotLSTM: Rotating Memories in Recurrent Neural Networks
arXiv:2105.00357
Abstract
Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.
References in corpus (7)
- LSTM: A Search Space Odyssey
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- End-To-End Memory Networks
- Deep Voice: Real-time Neural Text-to-Speech
- End-to-End Differentiable Proving
- Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs
- Sequence-to-sequence neural network models for transliteration