State Space LSTM Models with Particle MCMC Inference
arXiv:1711.11179
Abstract
Long Short-Term Memory (LSTM) is one of the most powerful sequence models. Despite the strong performance, however, it lacks the nice interpretability as in state space models. In this paper, we present a way to combine the best of both worlds by introducing State Space LSTM (SSL) models that generalizes the earlier work \cite{zaheer2017latent} of combining topic models with LSTM. However, unlike \cite{zaheer2017latent}, we do not make any factorization assumptions in our inference algorithm. We present an efficient sampler based on sequential Monte Carlo (SMC) method that draws from the joint posterior directly. Experimental results confirms the superiority and stability of this SMC inference algorithm on a variety of domains.
References in corpus (9)
- Auto-Encoding Variational Bayes
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Visualizing and Understanding Recurrent Networks
- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
- Learning Stochastic Recurrent Networks
- Sequential Neural Models with Stochastic Layers
- Deep Kalman Filters
- Black box variational inference for state space models
- Variational Sequential Monte Carlo
Cited by in corpus (9)
- KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
- Deep Factors for Forecasting
- Causal Inference for Time series Analysis: Problems, Methods and Evaluation
- Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies
- Forecasting Individualized Disease Trajectories using Interpretable Deep Learning
- Variational Dynamic Mixtures
- Relational State-Space Model for Stochastic Multi-Object Systems
- Deep Variational Luenberger-type Observer for Stochastic Video Prediction
- Switching Recurrent Kalman Networks