Self-Supervised 3D Keypoint Learning for Ego-motion Estimation
arXiv:1912.03426
Abstract
Detecting and matching robust viewpoint-invariant keypoints is critical for visual SLAM and Structure-from-Motion. State-of-the-art learning-based methods generate training samples via homography adaptation to create 2D synthetic views with known keypoint matches from a single image. This approach, however, does not generalize to non-planar 3D scenes with illumination variations commonly seen in real-world videos. In this work, we propose self-supervised learning of depth-aware keypoints directly from unlabeled videos. We jointly learn keypoint and depth estimation networks by combining appearance and geometric matching via a differentiable structure-from-motion module based on Procrustean residual pose correction. We describe how our self-supervised keypoints can be integrated into state-of-the-art visual odometry frameworks for robust and accurate ego-motion estimation of autonomous vehicles in real-world conditions.
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Faster and better: a machine learning approach to corner detection
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- FCOS: Fully Convolutional One-Stage Object Detection
- R2D2: Repeatable and Reliable Detector and Descriptor
- LF-Net: Learning Local Features from Images
- DeepV2D: Video to Depth with Differentiable Structure from Motion
- Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
- UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor
- Deep High-Resolution Representation Learning for Human Pose Estimation
- Self-Improving Visual Odometry
- Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera
- Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotion
- Two Stream Networks for Self-Supervised Ego-Motion Estimation
- PnP-Net: A hybrid Perspective-n-Point Network
- Quad-networks: unsupervised learning to rank for interest point detection
- Neural Outlier Rejection for Self-Supervised Keypoint Learning