Learning to Compare Image Patches via Convolutional Neural Networks

Abstract

In this paper we show how to learn directly from image data (i.e., without resorting to manually-designed features) a general similarity function for comparing image patches, which is a task of fundamental importance for many computer vision problems. To encode such a function, we opt for a CNN-based model that is trained to account for a wide variety of changes in image appearance. To that end, we explore and study multiple neural network architectures, which are specifically adapted to this task. We show that such an approach can significantly outperform the state-of-the-art on several problems and benchmark datasets.

CVPR 2015

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Intriguing properties of neural networks
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Computing the Stereo Matching Cost with a Convolutional Neural Network
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Cited by in corpus (91)

• Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
• Working hard to know your neighbor's margins: Local descriptor learning loss
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Universal Correspondence Network
• Rotation equivariant vector field networks
• Slice-to-volume medical image registration: a survey
• Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions
• PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors
• Siamese Instance Search for Tracking
• Learning to cluster in order to transfer across domains and tasks
• Review of data analysis in vision inspection of power lines with an in-depth discussion of deep learning technology
• Generic 3D Representation via Pose Estimation and Matching
• Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection
• Image similarity using Deep CNN and Curriculum Learning
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters
• Neural network-based clustering using pairwise constraints
• Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching
• Cross-Domain Image Matching with Deep Feature Maps
• Convolutional neural network architecture for geometric matching
• CliqueCNN: Deep Unsupervised Exemplar Learning
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Large-Scale Image Retrieval with Attentive Deep Local Features
• Smart Mining for Deep Metric Learning
• WarpNet: Weakly Supervised Matching for Single-view Reconstruction
• DeepMVS: Learning Multi-view Stereopsis
• SIFT Meets CNN: A Decade Survey of Instance Retrieval
• RankIQA: Learning from Rankings for No-reference Image Quality Assessment
• Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks
• A Deep Metric for Multimodal Registration
• Change Detection between Multimodal Remote Sensing Data Using Siamese CNN
• Localizing and Orienting Street Views Using Overhead Imagery
• EPI-based Oriented Relation Networks for Light Field Depth Estimation
• Cooperative Learning with Visual Attributes
• FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence
• StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction
• Local Feature Detectors, Descriptors, and Image Representations: A Survey
• Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
• An In-Depth Analysis of Visual Tracking with Siamese Neural Networks
• Fast Video Shot Transition Localization with Deep Structured Models
• Left-Right Comparative Recurrent Model for Stereo Matching
• Deep Online Video Stabilization
• Learning Spread-out Local Feature Descriptors
• Large-Scale Classification of Structured Objects using a CRF with Deep Class Embedding
• ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems
• Fast Feature Extraction with CNNs with Pooling Layers
• Adversarial Image Registration with Application for MR and TRUS Image Fusion
• Video2Shop: Exact Matching Clothes in Videos to Online Shopping Images
• Proxy Anchor Loss for Deep Metric Learning
• Deep Learning in Information Security
• End-to-End Training of Hybrid CNN-CRF Models for Stereo
• Matching Features without Descriptors: Implicitly Matched Interest Points
• A Performance Evaluation of Local Features for Image Based 3D Reconstruction
• Heterogeneous Graph Matching Networks
• Real-Time Dense Stereo Embedded in A UAV for Road Inspection
• Design Identification of Curve Patterns on Cultural Heritage Objects: Combining Template Matching and CNN-based Re-Ranking
• Sunrise or Sunset: Selective Comparison Learning for Subtle Attribute Recognition
• Spot the Difference by Object Detection
• Learning Dense Stereo Matching for Digital Surface Models from Satellite Imagery
• FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
• Deep Multi-Modal Image Correspondence Learning
• Deep Stereo Matching with Dense CRF Priors
• Optical Flow Requires Multiple Strategies (but only one network)
• Rare Event Detection using Disentangled Representation Learning
• Semantic Analysis of (Reflectional) Visual Symmetry: A Human-Centred Computational Model for Declarative Explainability
• AutoScaler: Scale-Attention Networks for Visual Correspondence
• Learning to Assign Orientations to Feature Points
• PatchBatch: a Batch Augmented Loss for Optical Flow
• Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling
• Siamese convolutional networks based on phonetic features for cognate identification
• DSR: Direct Self-rectification for Uncalibrated Dual-lens Cameras
• Visual-Interactive Similarity Search for Complex Objects by Example of Soccer Player Analysis
• Proposal Flow: Semantic Correspondences from Object Proposals
• Semantic Correspondence: A Hierarchical Approach
• An Efficient Approach for Geo-Multimedia Cross-Modal Retrieval
• Image Patch Matching Using Convolutional Descriptors with Euclidean Distance
• Learning Image Matching by Simply Watching Video
• TCDesc: Learning Topology Consistent Descriptors for Image Matching
• Weakly-Supervised Spatial Context Networks
• Attention-Mechanism-based Tracking Method for Intelligent Internet of Vehicles
• Online Descriptor Enhancement via Self-Labelling Triplets for Visual Data Association
• Camera Invariant Feature Learning for Generalized Face Anti-spoofing
• The Artificial Mind's Eye: Resisting Adversarials for Convolutional Neural Networks using Internal Projection
• Image Matching via Loopy RNN
• Fully Convolutional Siamese Networks for Change Detection
• Automatic Synchronization of Multi-User Photo Galleries
• Learning to Align Images using Weak Geometric Supervision
• DCI: Discriminative and Contrast Invertible Descriptor
• Deep Learning-Based Image Kernel for Inductive Transfer
• Matching Disparate Image Pairs Using Shape-Aware ConvNets
• Understanding and Improving Kernel Local Descriptors
• Learning Robust Video Synchronization without Annotations