DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction

Abstract

Dynamical models estimate and predict the temporal evolution of physical systems. State Space Models (SSMs) in particular represent the system dynamics with many desirable properties, such as being able to model uncertainty in both the model and measurements, and optimal (in the Bayesian sense) recursive formulations e.g. the Kalman Filter. However, they require significant domain knowledge to derive the parametric form and considerable hand-tuning to correctly set all the parameters. Data driven techniques e.g. Recurrent Neural Networks have emerged as compelling alternatives to SSMs with wide success across a number of challenging tasks, in part due to their ability to extract relevant features from rich inputs. They however lack interpretability and robustness to unseen conditions. In this work, we present DynaNet, a hybrid deep learning and time-varying state-space model which can be trained end-to-end. Our neural Kalman dynamical model allows us to exploit the relative merits of each approach. We demonstrate state-of-the-art estimation and prediction on a number of physically challenging tasks, including visual odometry, sensor fusion for visual-inertial navigation and pendulum control. In addition we show how DynaNet can indicate failures through investigation of properties such as the rate of innovation (Kalman Gain).

Accepted by IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

References in corpus (9)

• VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Learning to Navigate in Complex Environments
• BA-Net: Dense Bundle Adjustment Network
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Pathwise Derivatives Beyond the Reparameterization Trick
• Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic
• Particle Filter Networks with Application to Visual Localization
• Sparse Representations for Object and Ego-motion Estimation in Dynamic Scenes

Cited by in corpus (1)

• Ensemble Kalman Variational Objectives: Nonlinear Latent Trajectory Inference with A Hybrid of Variational Inference and Ensemble Kalman Filter