Temporally Folded Convolutional Neural Networks for Sequence Forecasting

Abstract

In this work we propose a novel approach to utilize convolutional neural networks for time series forecasting. The time direction of the sequential data with spatial dimensions $D=1,2$ is considered democratically as the input of a spatiotemporal $(D+1)$-dimensional convolutional neural network. Latter then reduces the data stream from $D+1\to D$ dimensions followed by an incriminator cell which uses this information to forecast the subsequent time step. We empirically compare this strategy to convolutional LSTM's and LSTM's on their performance on the sequential MNIST and the JSB chorals dataset, respectively. We conclude that temporally folded convolutional neural networks (TFC's) may outperform the conventional recurrent strategies.

arXiv comment8 pages, 4 figures, submitted to IJCAI 2020 Proceedings

References in corpus (15)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• LSTM: A Search Space Odyssey
• Deep Residual Learning for Image Recognition
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Convolutional Sequence to Sequence Learning
• Unsupervised Learning of Video Representations using LSTMs
• Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions
• Language Modeling with Gated Convolutional Networks
• Fully Convolutional Networks for Semantic Segmentation
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Neural Machine Translation in Linear Time
• How to Construct Deep Recurrent Neural Networks
• Architectural Complexity Measures of Recurrent Neural Networks
• Temporal Convolutional Networks for Action Segmentation and Detection