Do Convnets Learn Correspondence?

Abstract

Convolutional neural nets (convnets) trained from massive labeled datasets have substantially improved the state-of-the-art in image classification and object detection. However, visual understanding requires establishing correspondence on a finer level than object category. Given their large pooling regions and training from whole-image labels, it is not clear that convnets derive their success from an accurate correspondence model which could be used for precise localization. In this paper, we study the effectiveness of convnet activation features for tasks requiring correspondence. We present evidence that convnet features localize at a much finer scale than their receptive field sizes, that they can be used to perform intraclass alignment as well as conventional hand-engineered features, and that they outperform conventional features in keypoint prediction on objects from PASCAL VOC 2011.

References in corpus (4)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
• Learning Human Pose Estimation Features with Convolutional Networks

Cited by in corpus (29)

• Aggregating Deep Convolutional Features for Image Retrieval
• Slice-to-volume medical image registration: a survey
• Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation
• Spatial-Angular Attention Network for Light Field Reconstruction
• Neural Best-Buddies: Sparse Cross-Domain Correspondence
• Understanding Intra-Class Knowledge Inside CNN
• 6-DoF Object Pose from Semantic Keypoints
• A Recurrent Encoder-Decoder Network for Sequential Face Alignment
• Do Different Tracking Tasks Require Different Appearance Models?
• Cross-domain Correspondence Learning for Exemplar-based Image Translation
• From handcrafted to deep local features
• Dense Human Body Correspondences Using Convolutional Networks
• Learning Video Representations from Correspondence Proposals
• Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
• Semantic keypoint-based pose estimation from single RGB frames
• FeatherNets: Convolutional Neural Networks as Light as Feather for Face Anti-spoofing
• Correspondence Networks with Adaptive Neighbourhood Consensus
• CoKe: Localized Contrastive Learning for Robust Keypoint Detection
• Real-time Detection of 2D Tool Landmarks with Synthetic Training Data
• Learning Feature Descriptors using Camera Pose Supervision
• Global Semantic Description of Objects based on Prototype Theory
• Joint Viewpoint and Keypoint Estimation with Real and Synthetic Data
• Introducing the structural bases of typicality effects in deep learning
• Combating Ambiguity for Hash-code Learning in Medical Instance Retrieval
• Deep Lucas-Kanade Homography for Multimodal Image Alignment
• Low-Cost Transfer Learning of Face Tasks
• Unsupervised Part Discovery via Feature Alignment
• Multi-Scale Convolutions for Learning Context Aware Feature Representations
• DASNet: Reducing Pixel-level Annotations for Instance and Semantic Segmentation