Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces

Abstract

In order to integrate uncertainty estimates into deep time-series modelling, Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models, however, such approaches typically rely on approximate inference techniques such as variational inference which makes learning more complex and often less scalable due to approximation errors. We propose a new deep approach to Kalman filtering which can be learned directly in an end-to-end manner using backpropagation without additional approximations. Our approach uses a high-dimensional factorized latent state representation for which the Kalman updates simplify to scalar operations and thus avoids hard to backpropagate, computationally heavy and potentially unstable matrix inversions. Moreover, we use locally linear dynamic models to efficiently propagate the latent state to the next time step. The resulting network architecture, which we call Recurrent Kalman Network (RKN), can be used for any time-series data, similar to a LSTM (Hochreiter & Schmidhuber, 1997) but uses an explicit representation of uncertainty. As shown by our experiments, the RKN obtains much more accurate uncertainty estimates than an LSTM or Gated Recurrent Units (GRUs) (Cho et al., 2014) while also showing a slightly improved prediction performance and outperforms various recent generative models on an image imputation task.

accepted at ICML 2019

Cited by in corpus (13)

• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
• Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting
• Modeling Irregular Time Series with Continuous Recurrent Units
• Learning to Assimilate in Chaotic Dynamical Systems
• Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs
• Variational Dynamic Mixtures
• Action-Conditional Recurrent Kalman Networks For Forward and Inverse Dynamics Learning
• Learning Dynamical Systems from Noisy Sensor Measurements using Multiple Shooting
• Equivariant Deep Dynamical Model for Motion Prediction
• Switching Recurrent Kalman Networks
• Self-Supervised Inference in State-Space Models
• Deep Measurement Updates for Bayes Filters
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction