Unsupervised Learning of Monocular Depth Estimation with Bundle Adjustment, Super-Resolution and Clip Loss
arXiv:1812.03368
Abstract
We present a novel unsupervised learning framework for single view depth estimation using monocular videos. It is well known in 3D vision that enlarging the baseline can increase the depth estimation accuracy, and jointly optimizing a set of camera poses and landmarks is essential. In previous monocular unsupervised learning frameworks, only part of the photometric and geometric constraints within a sequence are used as supervisory signals. This may result in a short baseline and overfitting. Besides, previous works generally estimate a low resolution depth from a low resolution impute image. The low resolution depth is then interpolated to recover the original resolution. This strategy may generate large errors on object boundaries, as the depth of background and foreground are mixed to yield the high resolution depth. In this paper, we introduce a bundle adjustment framework and a super-resolution network to solve the above two problems. In bundle adjustment, depths and poses of an image sequence are jointly optimized, which increases the baseline by establishing the relationship between farther frames. The super resolution network learns to estimate a high resolution depth from a low resolution image. Additionally, we introduce the clip loss to deal with moving objects and occlusion. Experimental results on the KITTI dataset show that the proposed algorithm outperforms the state-of-the-art unsupervised methods using monocular sequences, and achieves comparable or even better result compared to unsupervised methods using stereo sequences.
References in corpus (22)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Deep Residual Learning for Image Recognition
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- DeMoN: Depth and Motion Network for Learning Monocular Stereo
- SfM-Net: Learning of Structure and Motion from Video
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
- Unsupervised Learning of Depth and Ego-Motion from Video
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
- Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
- Single-Image Depth Perception in the Wild
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction
- DeepStereo: Learning to Predict New Views from the World's Imagery
- Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
- MegaDepth: Learning Single-View Depth Prediction from Internet Photos
- Learning Depth from Monocular Videos using Direct Methods
- Learning Monocular Depth by Distilling Cross-domain Stereo Networks
- LEGO: Learning Edge with Geometry all at Once by Watching Videos
- Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding
- Learning monocular depth estimation with unsupervised trinocular assumptions