KFNet: Learning Temporal Camera Relocalization using Kalman Filtering
arXiv:2003.10629
Abstract
Temporal camera relocalization estimates the pose with respect to each video frame in sequence, as opposed to one-shot relocalization which focuses on a still image. Even though the time dependency has been taken into account, current temporal relocalization methods still generally underperform the state-of-the-art one-shot approaches in terms of accuracy. In this work, we improve the temporal relocalization method by using a network architecture that incorporates Kalman filtering (KFNet) for online camera relocalization. In particular, KFNet extends the scene coordinate regression problem to the time domain in order to recursively establish 2D and 3D correspondences for the pose determination. The network architecture design and the loss formulation are based on Kalman filtering in the context of Bayesian learning. Extensive experiments on multiple relocalization benchmarks demonstrate the high accuracy of KFNet at the top of both one-shot and temporal relocalization approaches. Our codes are released at https://github.com/zlthinker/KFNet.
arXiv commentAn oral paper of CVPR 2020
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
- Parallel Structure from Motion from Local Increment to Global Averaging
- VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
- Accurate Optical Flow via Direct Cost Volume Processing
- Improved Visual Relocalization by Discovering Anchor Points
- Understanding the Limitations of CNN-based Absolute Camera Pose Regression
- Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos
- Matchable Image Retrieval by Learning from Surface Reconstruction
- Scene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization