Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter

Abstract

Variational inference (VI) combined with Bayesian nonlinear filtering produces state-of-the-art results for latent time-series modeling. A body of recent work has focused on sequential Monte Carlo (SMC) and its variants, e.g., forward filtering backward simulation (FFBSi). Although these studies have succeeded, serious problems remain in particle degeneracy and biased gradient estimators. In this paper, we propose Ensemble Kalman Variational Objective (EnKO), a hybrid method of VI and the ensemble Kalman filter (EnKF), to infer state space models (SSMs). Our proposed method can efficiently identify latent dynamics because of its particle diversity and unbiased gradient estimators. We demonstrate that our EnKO outperforms SMC-based methods in terms of predictive ability and particle efficiency for three benchmark nonlinear system identification tasks.

References in corpus (20)

• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• DRAW: A Recurrent Neural Network For Image Generation
• A Recurrent Latent Variable Model for Sequential Data
• Learning Stochastic Recurrent Networks
• Deep Kalman Filters
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Black box variational inference for state space models
• ODE${}^2$VAE: Deep generative second order ODEs with Bayesian neural networks
• LFADS - Latent Factor Analysis via Dynamical Systems
• Auto-Encoding Sequential Monte Carlo
• Deep Temporal Sigmoid Belief Networks for Sequence Modeling
• Importance Weighting and Variational Inference
• Unbiased Implicit Variational Inference
• Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells
• Temporal Difference Variational Auto-Encoder
• Graphical model inference: Sequential Monte Carlo meets deterministic approximations
• DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
• Particle Smoothing Variational Objectives