Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps
arXiv:1907.05916 · doi:10.1145/3343031.3351020
Abstract
Recent works have shown Generative Adversarial Networks (GANs) to be particularly effective in image-to-image translations. However, in tasks such as body pose and hand gesture translation, existing methods usually require precise annotations, e.g. key-points or skeletons, which are time-consuming to draw. In this work, we propose a novel GAN architecture that decouples the required annotations into a category label - that specifies the gesture type - and a simple-to-draw category-independent conditional map - that expresses the location, rotation and size of the hand gesture. Our architecture synthesizes the target gesture while preserving the background context, thus effectively dealing with gesture translation in the wild. To this aim, we use an attention module and a rolling guidance approach, which loops the generated images back into the network and produces higher quality images compared to competing works. Thus, our GAN learns to generate new images from simple annotations without requiring key-points or skeleton labels. Results on two public datasets show that our method outperforms state of the art approaches both quantitatively and qualitatively. To the best of our knowledge, no work so far has addressed the gesture-to-gesture translation in the wild by requiring user-friendly annotations.
arXiv comment15 pages, 12 figures
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Empirical Evaluation of Rectified Activations in Convolutional Network
- NIPS 2016 Tutorial: Generative Adversarial Networks
- Toward Multimodal Image-to-Image Translation
- Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
- OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
- Pose Guided Person Image Generation
- A Note on the Inception Score
- Learning What and Where to Draw
- InstaGAN: Instance-aware Image-to-Image Translation
- GANimation: Anatomically-aware Facial Animation from a Single Image
- Triple consistency loss for pairing distributions in GAN-based face synthesis
- GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
- Skeleton-aided Articulated Motion Generation