Image Matching across Wide Baselines: From Paper to Practice

Abstract

We introduce a comprehensive benchmark for local features and robust estimation algorithms, focusing on the downstream task -- the accuracy of the reconstructed camera pose -- as our primary metric. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods and heuristics. This is demonstrated by embedding dozens of popular algorithms and evaluating them, from seminal works to the cutting edge of machine learning research. We show that with proper settings, classical solutions may still outperform the perceived state of the art. Besides establishing the actual state of the art, the conducted experiments reveal unexpected properties of Structure from Motion (SfM) pipelines that can help improve their performance, for both algorithmic and learned methods. Data and code are online https://github.com/vcg-uvic/image-matching-benchmark, providing an easy-to-use and flexible framework for the benchmarking of local features and robust estimation methods, both alongside and against top-performing methods. This work provides a basis for the Image Matching Challenge https://vision.uvic.ca/image-matching-challenge.

arXiv commentAdded: KeyNet-SOSNet, AffNet-HardNet, TFeat, MKD from kornia

References in corpus (14)

• ORB-SLAM: a Versatile and Accurate Monocular SLAM System
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Faster and better: a machine learning approach to corner detection
• YFCC100M: The New Data in Multimedia Research
• R2D2: Repeatable and Reliable Detector and Descriptor
• SfM-Net: Learning of Structure and Motion from Video
• TILDE: A Temporally Invariant Learned DEtector
• Working hard to know your neighbor's margins: Local descriptor learning loss
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
• MODS: Fast and Robust Method for Two-View Matching
• InLoc: Indoor Visual Localization with Dense Matching and View Synthesis
• An Evaluation of Feature Matchers for Fundamental Matrix Estimation
• Leveraging Outdoor Webcams for Local Descriptor Learning
• Self-Supervised Learning of Depth and Ego-motion with Differentiable Bundle Adjustment

Cited by in corpus (24)

• VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change
• ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning
• Tracking and Mapping in Medical Computer Vision: A Review
• SIFT Matching by Context Exposed
• Parallel Structure from Motion for UAV Images via Weighted Connected Dominating Set
• Feature matching for multi-epoch historical aerial images
• Advanced computer vision for extracting georeferenced vehicle trajectories from drone imagery
• HarrisZ${}^{+}$: Harris Corner Selection for Next-Gen Image Matching Pipelines
• DAOT: Domain-Agnostically Aligned Optimal Transport for Domain-Adaptive Crowd Counting
• Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation
• Deep Multi-View Stereo gone wild
• SuperPoint features in endoscopy
• TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching
• Illumination-insensitive Binary Descriptor for Visual Measurement Based on Local Inter-patch Invariance
• Image Matching by Bare Homography
• Improving the matching of deformable objects by learning to detect keypoints
• A2B: Anchor to Barycentric Coordinate for Robust Correspondence
• Learning Geodesic-Aware Local Features from RGB-D Images
• NeSS-ST: Detecting Good and Stable Keypoints with a Neural Stability Score and the Shi-Tomasi Detector
• Robust Self-calibration of Focal Lengths from the Fundamental Matrix
• A Survey of Feature detection methods for localisation of plain sections of Axial Brain Magnetic Resonance Imaging
• CNN-based local features for navigation near an asteroid
• Are Minimal Radial Distortion Solvers Necessary for Relative Pose Estimation?
• Optimal detection of the feature matching map in presence of noise and outliers