Weakly-Supervised Spatial Context Networks
arXiv:1704.02998
Abstract
We explore the power of spatial context as a self-supervisory signal for learning visual representations. In particular, we propose spatial context networks that learn to predict a representation of one image patch from another image patch, within the same image, conditioned on their real-valued relative spatial offset. Unlike auto-encoders, that aim to encode and reconstruct original image patches, our network aims to encode and reconstruct intermediate representations of the spatially offset patches. As such, the network learns a spatially conditioned contextual representation. By testing performance with various patch selection mechanisms we show that focusing on object-centric patches is important, and that using object proposal as a patch selection mechanism leads to the highest improvement in performance. Further, unlike auto-encoders, context encoders [21], or other forms of unsupervised feature learning, we illustrate that contextual supervision (with pre-trained model initialization) can improve on existing pre-trained model performance. We build our spatial context networks on top of standard VGG_19 and CNN_M architectures and, among other things, show that we can achieve improvements (with no additional explicit supervision) over the original ImageNet pre-trained VGG_19 and CNN_M models in object categorization and detection on VOC2007.
References in corpus (8)
- Efficient Estimation of Word Representations in Vector Space
- Distilling the Knowledge in a Neural Network
- DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
- CNN Features off-the-shelf: an Astounding Baseline for Recognition
- Unsupervised Visual Representation Learning by Context Prediction
- Learning to Compare Image Patches via Convolutional Neural Networks
- Unsupervised Learning of Visual Representations using Videos
- Learning to See by Moving