GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints

Abstract

Learned local descriptors based on Convolutional Neural Networks (CNNs) have achieved significant improvements on patch-based benchmarks, whereas not having demonstrated strong generalization ability on recent benchmarks of image-based 3D reconstruction. In this paper, we mitigate this limitation by proposing a novel local descriptor learning approach that integrates geometry constraints from multi-view reconstructions, which benefits the learning process in terms of data generation, data sampling and loss computation. We refer to the proposed descriptor as GeoDesc, and demonstrate its superior performance on various large-scale benchmarks, and in particular show its great success on challenging reconstruction tasks. Moreover, we provide guidelines towards practical integration of learned descriptors in Structure-from-Motion (SfM) pipelines, showing the good trade-off that GeoDesc delivers to 3D reconstruction tasks between accuracy and efficiency.

arXiv commentAccepted to ECCV'18

References in corpus (6)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Working hard to know your neighbor's margins: Local descriptor learning loss
• Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
• DeepHash: Getting Regularization, Depth and Fine-Tuning Right
• A Large Dataset for Improving Patch Matching

Cited by in corpus (40)

• Image Matching across Wide Baselines: From Paper to Practice
• DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
• DISK: Learning local features with policy gradient
• Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, Applications, and Prospects
• Local Feature Matching Using Deep Learning: A Survey
• Learning Two-View Correspondences and Geometry Using Order-Aware Network
• SIFT Matching by Context Exposed
• An Evaluation of Feature Matchers for Fundamental Matrix Estimation
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features
• BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks
• ContextDesc: Local Descriptor Augmentation with Cross-Modality Context
• PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency
• HyNet: Learning Local Descriptor with Hybrid Similarity Measure and Triplet Loss
• Improving the HardNet Descriptor
• Learnable Motion Coherence for Correspondence Pruning
• Leveraging Outdoor Webcams for Local Descriptor Learning
• Matchable Image Retrieval by Learning from Surface Reconstruction
• AdaSample: Adaptive Sampling of Hard Positives for Descriptor Learning
• High-Precision Localization Using Ground Texture
• Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation
• iMatching: Imperative Correspondence Learning
• Dense Semantic 3D Map Based Long-Term Visual Localization with Hybrid Features
• Extremely Dense Point Correspondences using a Learned Feature Descriptor
• End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds
• D2D: Keypoint Extraction with Describe to Detect Approach
• RANSAC-Flow: generic two-stage image alignment
• Learning Feature Descriptors using Camera Pose Supervision
• Extracting Deformation-Aware Local Features by Learning to Deform
• SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence
• SDGMNet: Statistic-based Dynamic Gradient Modulation for Local Descriptor Learning
• KFNet: Learning Temporal Camera Relocalization using Kalman Filtering
• CNN-based local features for navigation near an asteroid
• Stochastic Bundle Adjustment for Efficient and Scalable 3D Reconstruction
• Deep Homography Estimation for Dynamic Scenes
• Efficient Spatially Adaptive Convolution and Correlation
• IF-Net: An Illumination-invariant Feature Network
• An End to End Network Architecture for Fundamental Matrix Estimation
• Learning Local Feature Descriptor with Motion Attribute for Vision-based Localization
• Leveraging Local and Global Descriptors in Parallel to Search Correspondences for Visual Localization
• Compression of descriptor models for mobile applications