Variational Tracking and Prediction with Generative Disentangled State-Space Models

Abstract

We address tracking and prediction of multiple moving objects in visual data streams as inference and sampling in a disentangled latent state-space model. By encoding objects separately and including explicit position information in the latent state space, we perform tracking via amortized variational Bayesian inference of the respective latent positions. Inference is implemented in a modular neural framework tailored towards our disentangled latent space. Generative and inference model are jointly learned from observations only. Comparing to related prior work, we empirically show that our Markovian state-space assumption enables faithful and much improved long-term prediction well beyond the training horizon. Further, our inference model correctly decomposes frames into objects, even in the presence of occlusions. Tracking performance is increased significantly over prior art.

References in corpus (8)

• Sequence to Sequence Learning with Neural Networks
• Simple Online and Realtime Tracking
• Unsupervised Learning of Video Representations using LSTMs
• A Recurrent Latent Variable Model for Sequential Data
• Stochastic Video Generation with a Learned Prior
• Learning Stochastic Recurrent Networks
• Using Fast Weights to Attend to the Recent Past
• Black box variational inference for state space models

Cited by in corpus (2)

• Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models
• Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF