UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor

Abstract

It is hard to create consistent ground truth data for interest points in natural images, since interest points are hard to define clearly and consistently for a human annotator. This makes interest point detectors non-trivial to build. In this work, we introduce an unsupervised deep learning-based interest point detector and descriptor. Using a self-supervised approach, we utilize a siamese network and a novel loss function that enables interest point scores and positions to be learned automatically. The resulting interest point detector and descriptor is UnsuperPoint. We use regression of point positions to 1) make UnsuperPoint end-to-end trainable and 2) to incorporate non-maximum suppression in the model. Unlike most trainable detectors, it requires no generation of pseudo ground truth points, no structure-from-motion-generated representations and the model is learned from only one round of training. Furthermore, we introduce a novel loss function to regularize network predictions to be uniformly distributed. UnsuperPoint runs in real-time with 323 frames per second (fps) at a resolution of $224\times 320$ and 90 fps at $480\times 640$. It is comparable or better than state-of-the-art performance when measured for speed, repeatability, localization, matching score and homography estimation on the HPatch dataset.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
• Densely Connected Convolutional Networks
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• TILDE: A Temporally Invariant Learned DEtector
• Deep Image Homography Estimation
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors

Cited by in corpus (10)

• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features
• Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines
• Self-Supervised 3D Keypoint Learning for Ego-motion Estimation
• D2D: Keypoint Extraction with Describe to Detect Approach
• Learning Feature Descriptors using Camera Pose Supervision
• Neural Outlier Rejection for Self-Supervised Keypoint Learning
• Realtime CNN-based Keypoint Detector with Sobel Filter and CNN-based Descriptor Trained with Keypoint Candidates
• SOLD2: Self-supervised Occlusion-aware Line Description and Detection
• Soft Expectation and Deep Maximization for Image Feature Detection