WarpNet: Weakly Supervised Matching for Single-view Reconstruction

Abstract

We present an approach to matching images of objects in fine-grained datasets without using part annotations, with an application to the challenging problem of weakly supervised single-view reconstruction. This is in contrast to prior works that require part annotations, since matching objects across class and pose variations is challenging with appearance features alone. We overcome this challenge through a novel deep learning architecture, WarpNet, that aligns an object in one image with a different object in another. We exploit the structure of the fine-grained dataset to create artificial data for training this network in an unsupervised-discriminative learning approach. The output of the network acts as a spatial prior that allows generalization at test time to match real images across variations in appearance, viewpoint and articulation. On the CUB-200-2011 dataset of bird categories, we improve the AP over an appearance-only network by 13.6%. We further demonstrate that our WarpNet matches, together with the structure of fine-grained datasets, allow single-view reconstructions with quality comparable to using annotated point correspondences.

to appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016

References in corpus (6)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Learning to Compare Image Patches via Convolutional Neural Networks
• Learning to See by Moving
• Viewpoints and Keypoints
• Virtual View Networks for Object Reconstruction

Cited by in corpus (19)

• Convolutional neural network architecture for geometric matching
• Recurrent Transformer Networks for Semantic Correspondence
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• Skin Cancer Detection and Tracking using Data Synthesis and Deep Learning
• VITON: An Image-based Virtual Try-on Network
• ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning
• Semantic Video Segmentation by Gated Recurrent Flow Propagation
• Siamese Encoding and Alignment by Multiscale Learning with Self-Supervision
• Unsupervised learning of object frames by dense equivariant image labelling
• DSAC - Differentiable RANSAC for Camera Localization
• Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing
• 3D Reconstruction of Simple Objects from A Single View Silhouette Image
• PixelNN: Example-based Image Synthesis
• GridFace: Face Rectification via Learning Local Homography Transformations
• Multi-Image Semantic Matching by Mining Consistent Features
• Statistical transformer networks: learning shape and appearance models via self supervision
• Adversarial Learning for Image Forensics Deep Matching with Atrous Convolution
• W-TALC: Weakly-supervised Temporal Activity Localization and Classification
• Learning Robust Video Synchronization without Annotations