Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry
arXiv:1908.08704
Abstract
We propose a self-supervised learning framework for visual odometry (VO) that incorporates correlation of consecutive frames and takes advantage of adversarial learning. Previous methods tackle self-supervised VO as a local structure from motion (SfM) problem that recovers depth from single image and relative poses from image pairs by minimizing photometric loss between warped and captured images. As single-view depth estimation is an ill-posed problem, and photometric loss is incapable of discriminating distortion artifacts of warped images, the estimated depth is vague and pose is inaccurate. In contrast to previous methods, our framework learns a compact representation of frame-to-frame correlation, which is updated by incorporating sequential information. The updated representation is used for depth estimation. Besides, we tackle VO as a self-supervised image generation task and take advantage of Generative Adversarial Networks (GAN). The generator learns to estimate depth and pose to generate a warped target image. The discriminator evaluates the quality of generated image with high-level structural perception that overcomes the problem of pixel-wise loss in previous methods. Experiments on KITTI and Cityscapes datasets show that our method obtains more accurate depth with details preserved and predicted pose outperforms state-of-the-art self-supervised methods significantly.
Accept to ICCV 2019
References in corpus (12)
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- DeMoN: Depth and Motion Network for Learning Monocular Stereo
- FlowNet: Learning Optical Flow with Convolutional Networks
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
- Unsupervised Learning of Depth and Ego-Motion from Video
- GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
- Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
- Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry
- A Deeper Insight into the UnDEMoN: Unsupervised Deep Network for Depth and Ego-Motion Estimation