Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Abstract

Latest results indicate that features learned via convolutional neural networks outperform previous descriptors on classification tasks by a large margin. It has been shown that these networks still work well when they are applied to datasets or recognition tasks different from those they were trained on. However, descriptors like SIFT are not only used in recognition but also for many correspondence problems that rely on descriptor matching. In this paper we compare features from various layers of convolutional neural nets to standard SIFT descriptors. We consider a network that was trained on ImageNet and another one that was trained without supervision. Surprisingly, convolutional neural networks clearly outperform SIFT on descriptor matching. This paper has been merged with arXiv:1406.6909

arXiv commentThis paper has been merged with arXiv:1406.6909

References in corpus (1)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

Cited by in corpus (42)

• Fully Convolutional Networks for Semantic Segmentation
• FlowNet: Learning Optical Flow with Convolutional Networks
• A Survey on Deep Learning Techniques for Stereo-based Depth Estimation
• Convolutional Neural Network-based Place Recognition
• Learning to Compare Image Patches via Convolutional Neural Networks
• LSDA: Large Scale Detection Through Adaptation
• Do Convnets Learn Correspondence?
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
• Generic 3D Representation via Pose Estimation and Matching
• Neural Best-Buddies: Sparse Cross-Domain Correspondence
• Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks
• SIFT Meets CNN: A Decade Survey of Instance Retrieval
• Visual Representations: Defining Properties and Deep Approximations
• Place recognition: An Overview of Vision Perspective
• What is Holding Back Convnets for Detection?
• From handcrafted to deep local features
• Large scale evaluation of local image feature detectors on homography datasets
• FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
• A Large Dataset for Improving Patch Matching
• DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features
• ReHAR: Robust and Efficient Human Activity Recognition
• Visual-Inertial-Semantic Scene Representation for 3-D Object Detection
• Domain-Size Pooling in Local Descriptors: DSP-SIFT
• Multi-Image Semantic Matching by Mining Consistent Features
• Unsupervised Abnormality Detection Using Heterogeneous Autonomous Systems
• Visual Scene Representations: Contrast, Scaling and Occlusion
• What Do Deep CNNs Learn About Objects?
• End-to-End Face Parsing via Interlinked Convolutional Neural Networks
• Persistent Evidence of Local Image Properties in Generic ConvNets
• Matching Underwater Sonar Images by the Learned Descriptor Based on Style Transfer Method
• Learning Image Matching by Simply Watching Video
• Semi-supervised learning of deep metrics for stereo reconstruction
• Improved Descriptors for Patch Matching and Reconstruction
• Fine-Grained Texture Identification for Reliable Product Traceability
• Image Matching via Loopy RNN
• DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation
• Training Deep Neural Networks to Detect Repeatable 2D Features Using Large Amounts of 3D World Capture Data
• Learning Transformation-Aware Embeddings for Image Forensics
• Robust Angular Local Descriptor Learning
• Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art
• PicHunt: Social Media Image Retrieval for Improved Law Enforcement
• Online Descriptor Enhancement via Self-Labelling Triplets for Visual Data Association