Disentangling Latent Hands for Image Synthesis and Pose Estimation

Abstract

Hand image synthesis and pose estimation from RGB images are both highly challenging tasks due to the large discrepancy between factors of variation ranging from image background content to camera viewpoint. To better analyze these factors of variation, we propose the use of disentangled representations and a disentangled variational autoencoder (dVAE) that allows for specific sampling and inference of these factors. The derived objective from the variational lower bound as well as the proposed training strategy are highly flexible, allowing us to handle cross-modal encoders and decoders as well as semi-supervised learning scenarios. Experiments show that our dVAE can synthesize highly realistic images of the hand specifiable by both pose and image background content and also estimate 3D hand poses from RGB images with accuracy competitive with state-of-the-art on two public benchmarks.

CVPR 2019

References in corpus (8)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Convolutional Inverse Graphics Network
• Hands Deep in Deep Learning for Hand Pose Estimation
• Capturing Hands in Action using Discriminative Salient Points and Physics Simulation
• Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
• 3D Hand Pose Tracking and Estimation Using Stereo Matching
• Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders
• Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization

Cited by in corpus (2)

• Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
• Disentangling Pose from Appearance in Monochrome Hand Images