Multi-variable LSTM neural network for autoregressive exogenous model

Abstract

In this paper, we propose multi-variable LSTM capable of accurate forecasting and variable importance interpretation for time series with exogenous variables. Current attention mechanism in recurrent neural networks mostly focuses on the temporal aspect of data and falls short of characterizing variable importance. To this end, the multi-variable LSTM equipped with tensorized hidden states is developed to learn hidden states for individual variables, which give rise to our mixture temporal and variable attention. Based on such attention mechanism, we infer and quantify variable importance. Extensive experiments using real datasets with Granger-causality test and the synthetic dataset with ground truth demonstrate the prediction performance and interpretability of multi-variable LSTM in comparison to a variety of baselines. It exhibits the prospect of multi-variable LSTM as an end-to-end framework for both forecasting and knowledge discovery.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Tensorizing Neural Networks
• Learning to Diagnose with LSTM Recurrent Neural Networks
• A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
• A Clockwork RNN
• Factorization tricks for LSTM networks
• Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
• Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning
• Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies
• Learning Deep Matrix Representations

Cited by in corpus (2)

• Bitcoin Volatility Forecasting with a Glimpse into Buy and Sell Orders
• A Memory-Network Based Solution for Multivariate Time-Series Forecasting