Learning Image Matching by Simply Watching Video
arXiv:1603.06041
Abstract
This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching. We start from the insight that the problem of frame-interpolation implicitly solves for inter-frame correspondences. This permits the application of analysis-by-synthesis: we firstly train and apply a Convolutional Neural Network for frame-interpolation, then obtain correspondences by inverting the learned CNN. The key benefit behind this strategy is that the CNN for frame-interpolation can be trained in an unsupervised manner by exploiting the temporal coherency that is naturally contained in real-world video sequences. The present model therefore learns image matching by simply watching videos. Besides a promise to be more generally applicable, the presented approach achieves surprising performance comparable to traditional empirically designed methods.
The second version contains additional quantitative evaluation of frame interpolation
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- Computing the Stereo Matching Cost with a Convolutional Neural Network
- MatConvNet - Convolutional Neural Networks for MATLAB
- FlowNet: Learning Optical Flow with Convolutional Networks
- Video (language) modeling: a baseline for generative models of natural videos
- Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT
- Learning to Compare Image Patches via Convolutional Neural Networks
- Learning to See by Moving
- Understanding Deep Image Representations by Inverting Them
- DeepMatching: Hierarchical Deformable Dense Matching