Exploit Clues from Views: Self-Supervised and Regularized Learning for Multiview Object Recognition

Abstract

Multiview recognition has been well studied in the literature and achieves decent performance in object recognition and retrieval task. However, most previous works rely on supervised learning and some impractical underlying assumptions, such as the availability of all views in training and inference time. In this work, the problem of multiview self-supervised learning (MV-SSL) is investigated, where only image to object association is given. Given this setup, a novel surrogate task for self-supervised learning is proposed by pursuing "object invariant" representation. This is solved by randomly selecting an image feature of an object as object prototype, accompanied with multiview consistency regularization, which results in view invariant stochastic prototype embedding (VISPE). Experiments shows that the recognition and retrieval results using VISPE outperform that of other self-supervised learning methods on seen and unseen data. VISPE can also be applied to semi-supervised scenario and demonstrates robust performance with limited data available. Code is available at https://github.com/chihhuiho/VISPE

arXiv commentAccepted to CVPR2020

References in corpus (17)

• Prototypical Networks for Few-shot Learning
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• Unsupervised Learning of Visual Representations using Videos
• Mitigating Adversarial Effects Through Randomization
• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Improvements to context based self-supervised learning
• Learning Representations for Automatic Colorization
• Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
• No Fuss Distance Metric Learning using Proxies
• Colorization as a Proxy Task for Visual Understanding
• Triplet-Center Loss for Multi-View 3D Object Retrieval
• PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
• Learning Image Representations by Completing Damaged Jigsaw Puzzles
• Learning Features by Watching Objects Move
• Self-Supervised Relative Depth Learning for Urban Scene Understanding
• Order Matters: Shuffling Sequence Generation for Video Prediction
• Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action Recognition