Latent Transformations for Object View Points Synthesis

Abstract

We propose a fully-convolutional conditional generative model, the latent transformation neural network (LTNN), capable of view synthesis using a light-weight neural network suited for real-time applications. In contrast to existing conditional generative models which incorporate conditioning information via concatenation, we introduce a dedicated network component, the conditional transformation unit (CTU), designed to learn the latent space transformations corresponding to specified target views. In addition, a consistency loss term is defined to guide the network toward learning the desired latent space mappings, a task-divided decoder is constructed to refine the quality of generated views, and an adaptive discriminator is introduced to improve the adversarial training process. The generality of the proposed methodology is demonstrated on a collection of three diverse tasks: multi-view reconstruction on real hand depth images, view synthesis of real and synthetic faces, and the rotation of rigid objects. The proposed model is shown to exceed state-of-the-art results in each category while simultaneously achieving a reduction in the computational demand required for inference by 30% on average.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Conditional Generative Adversarial Nets
• Densely Connected Convolutional Networks
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• Searching for Activation Functions
• Deep Convolutional Inverse Graphics Network
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• cGANs with Projection Discriminator
• Least Squares Generative Adversarial Networks
• Age Progression/Regression by Conditional Adversarial Autoencoder
• CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training
• Shape Completion Enabled Robotic Grasping