Particle Filter Recurrent Neural Networks
arXiv:1905.12885
Abstract
Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and noisy real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.
arXiv commentAccepted to AAAI 2020
References in corpus (13)
- Auto-Encoding Variational Bayes
- The Microsoft 2017 Conversational Speech Recognition System
- Light Gated Recurrent Units for Speech Recognition
- A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
- Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
- Bayesian Recurrent Neural Networks
- Online Decorrelation of Humidity and Temperature in Chemical Sensors for Continuous Monitoring
- Backprop KF: Learning Discriminative Deterministic State Estimators
- Auto-Encoding Sequential Monte Carlo
- Relational recurrent neural networks
- Deep Variational Reinforcement Learning for POMDPs
- Text classification based on ensemble extreme learning machine
- Temporal Difference Variational Auto-Encoder