Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting
arXiv:2101.12072
Abstract
In this work, we propose \texttt{TimeGrad}, an autoregressive model for multivariate probabilistic time series forecasting which samples from the data distribution at each time step by estimating its gradient. To this end, we use diffusion probabilistic models, a class of latent variable models closely connected to score matching and energy-based methods. Our model learns gradients by optimizing a variational bound on the data likelihood and at inference time converts white noise into a sample of the distribution of interest through a Markov chain using Langevin sampling. We demonstrate experimentally that the proposed autoregressive denoising diffusion model is the new state-of-the-art multivariate probabilistic forecasting method on real-world data sets with thousands of correlated dimensions. We hope that this method is a useful tool for practitioners and lays the foundation for future research in this area.
References in corpus (12)
- Sequence to Sequence Learning with Neural Networks
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Denoising Diffusion Probabilistic Models
- WaveNet: A Generative Model for Raw Audio
- Deep Unsupervised Learning using Nonequilibrium Thermodynamics
- Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
- Generative Modeling by Estimating Gradients of the Data Distribution
- Density estimation using Real NVP
- Deep and Confident Prediction for Time Series at Uber
- Normalizing Flows for Probabilistic Modeling and Inference
- Improved Techniques for Training Score-Based Generative Models
- Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows