Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry

Abstract

We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets to facilitate knowledge transfer tasks.

The version of IEEE Transactions on Pattern Analysis and Machine Intelligence

References in corpus (9)

• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Learning Deep Representation for Face Alignment with Auxiliary Attributes
• Deep Variational Information Bottleneck
• Disentangling factors of variation in deep representations using adversarial training
• Neural Variational Inference and Learning in Belief Networks
• Flexibly Fair Representation Learning by Disentanglement
• Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance

Cited by in corpus (6)

• Explicit Disentanglement of Appearance and Perspective in Generative Models
• Learning Feature-to-Feature Translator by Alternating Back-Propagation for Generative Zero-Shot Learning
• Learning Dynamic Generator Model by Alternating Back-Propagation Through Time
• MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation
• Disentangled Deep Autoencoding Regularization for Robust Image Classification
• Inducing Sparse Coding and And-Or Grammar from Generator Network