Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models

Abstract

Given large amount of real photos for training, Convolutional neural network shows excellent performance on object recognition tasks. However, the process of collecting data is so tedious and the background are also limited which makes it hard to establish a perfect database. In this paper, our generative model trained with synthetic images rendered from 3D models reduces the workload of data collection and limitation of conditions. Our structure is composed of two sub-networks: semantic foreground object reconstruction network based on Bayesian inference and classification network based on multi-triplet cost function for avoiding over-fitting problem on monotone surface and fully utilizing pose information by establishing sphere-like distribution of descriptors in each category which is helpful for recognition on regular photos according to poses, lighting condition, background and category information of rendered images. Firstly, our conjugate structure called generative model with metric learning utilizing additional foreground object channels generated from Bayesian rendering as the joint of two sub-networks. Multi-triplet cost function based on poses for object recognition are used for metric learning which makes it possible training a category classifier purely based on synthetic data. Secondly, we design a coordinate training strategy with the help of adaptive noises acting as corruption on input images to help both sub-networks benefit from each other and avoid inharmonious parameter tuning due to different convergence speed of two sub-networks. Our structure achieves the state of the art accuracy of over 50\% on ShapeNet database with data migration obstacle from synthetic images to real photos. This pipeline makes it applicable to do recognition on real images only based on 3D models.

14 pages

References in corpus (10)

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Variational Inference: A Review for Statisticians
• Fully Convolutional Networks for Semantic Segmentation
• ShapeNet: An Information-Rich 3D Model Repository
• Semi-Supervised Learning with Deep Generative Models
• Going Deeper with Convolutions
• Learning Deconvolution Network for Semantic Segmentation
• Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• Learning Fine-grained Image Similarity with Deep Ranking