BA-Net: Dense Bundle Adjustment Network
arXiv:1806.04807
Abstract
This paper introduces a network architecture to solve the structure-from-motion (SfM) problem via feature-metric bundle adjustment (BA), which explicitly enforces multi-view geometry constraints in the form of feature-metric error. The whole pipeline is differentiable so that the network can learn suitable features that make the BA problem more tractable. Furthermore, this work introduces a novel depth parameterization to recover dense per-pixel depth. The network first generates several basis depth maps according to the input image and optimizes the final depth as a linear combination of these basis depth maps via feature-metric BA. The basis depth maps generator is also learned via end-to-end training. The whole system nicely combines domain knowledge (i.e. hard-coded multi-view geometry constraints) and deep learning (i.e. feature learning and basis depth maps learning) to address the challenging dense SfM problem. Experiments on large scale real data prove the success of the proposed method.
arXiv commentRevised for rebuttal Code available at https://github.com/frobelbest/BANet
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- DeMoN: Depth and Motion Network for Learning Monocular Stereo
- Holistically-Nested Edge Detection
- End-to-End Learning of Geometry and Context for Deep Stereo Regression
- A Photometrically Calibrated Benchmark For Monocular Visual Odometry
- The BerHu penalty and the grouped effect
Cited by in corpus (40)
- DeepV2D: Video to Depth with Differentiable Structure from Motion
- RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
- A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
- On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation
- Tracking and Mapping in Medical Computer Vision: A Review
- Bimodal Camera Pose Prediction for Endoscopy
- Neural Non-Rigid Tracking
- Deep Online Correction for Monocular Visual Odometry
- Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness
- Learning models for visual 3D localization with implicit mapping
- RAGO: Recurrent Graph Optimizer For Multiple Rotation Averaging
- RegNet: Learning the Optimization of Direct Image-to-Image Pose Registration
- Unsupervised Visual Odometry and Action Integration for PointGoal Navigation in Indoor Environment
- Consistent Video Depth Estimation
- Self-Supervised Learning of Depth and Ego-motion with Differentiable Bundle Adjustment
- Towards Better Generalization: Joint Depth-Pose Learning without PoseNet
- DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
- DMLO: Deep Matching LiDAR Odometry
- VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enough
- Point-Based Multi-View Stereo Network
- Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation
- TartanVO: A Generalizable Learning-based VO
- PnP-Net: A hybrid Perspective-n-Point Network
- CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
- Optimizing Through Learned Errors for Accurate Sports Field Registration
- DeepSFM: Structure From Motion Via Deep Bundle Adjustment
- TransCamP: Graph Transformer for 6-DoF Camera Pose Estimation
- Cutting-edge 3D reconstruction solutions for underwater coral reef images: A review and comparison
- Learning to Optimize Non-Rigid Tracking
- Unsupervised Learning of Camera Pose with Compositional Re-estimation
- Unsupervised Learning of Lidar Features for Use in a Probabilistic Trajectory Estimator
- Depth Completion via Deep Basis Fitting
- Tangent Space Backpropagation for 3D Transformation Groups
- Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation
- Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty
- 3D Pipe Network Reconstruction Based on Structure from Motion with Incremental Conic Shape Detection and Cylindrical Constraint
- Deep Two-View Structure-from-Motion Revisited
- Consistency Guided Scene Flow Estimation
- VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals
- Deep Probabilistic Feature-metric Tracking