AutoScaler: Scale-Attention Networks for Visual Correspondence
arXiv:1611.05837
Abstract
Finding visual correspondence between local features is key to many computer vision problems. While defining features with larger contextual scales usually implies greater discriminativeness, it could also lead to less spatial accuracy of the features. We propose AutoScaler, a scale-attention network to explicitly optimize this trade-off in visual correspondence tasks. Our network consists of a weight-sharing feature network to compute multi-scale feature maps and an attention network to combine them optimally in the scale space. This allows our network to have adaptive receptive field sizes over different scales of the input. The entire network is trained end-to-end in a siamese framework for visual correspondence tasks. Our method achieves favorable results compared to state-of-the-art methods on challenging optical flow and semantic matching benchmarks, including Sintel, KITTI and CUB-2011. We also show that our method can generalize to improve hand-crafted descriptors (e.g Daisy) on general visual correspondence tasks. Finally, our attention network can generate visually interpretable scale attention maps.
References in corpus (11)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Multi-Scale Context Aggregation by Dilated Convolutions
- Going Deeper with Convolutions
- DRAW: A Recurrent Neural Network For Image Generation
- Computing the Stereo Matching Cost with a Convolutional Neural Network
- Fully Convolutional Networks for Semantic Segmentation
- Multiple Object Recognition with Visual Attention
- Rich feature hierarchies for accurate object detection and semantic segmentation
- Universal Correspondence Network
- Learning to Compare Image Patches via Convolutional Neural Networks