Learning Over Long Time Lags

Abstract

The advantage of recurrent neural networks (RNNs) in learning dependencies between time-series data has distinguished RNNs from other deep learning models. Recently, many advances are proposed in this emerging field. However, there is a lack of comprehensive review on memory models in RNNs in the literature. This paper provides a fundamental review on RNNs and long short term memory (LSTM) model. Then, provides a surveys of recent advances in different memory enhancements and learning techniques for capturing long term dependencies in RNNs.

This is a draft article, in preparation to submit for peer-review

References in corpus (4)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Deep Learning with S-shaped Rectified Linear Activation Units

Cited by in corpus (1)

• Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification