Deep Image Spatial Transformation for Person Image Generation
arXiv:2003.00696
Abstract
Pose-guided person image generation is to transform a source person image to a target pose. This task requires spatial manipulations of source data. However, Convolutional Neural Networks are limited by the lack of ability to spatially transform the inputs. In this paper, we propose a differentiable global-flow local-attention framework to reassemble the inputs at the feature level. Specifically, our model first calculates the global correlations between sources and targets to predict flow fields. Then, the flowed local patch pairs are extracted from the feature maps to calculate the local attention coefficients. Finally, we warp the source features using a content-aware sampling method with the obtained local attention coefficients. The results of both subjective and objective experiments demonstrate the superiority of our model. Besides, additional results in video animation and view synthesis show that our model is applicable to other tasks requiring spatial transformation. Our source code is available at https://github.com/RenYurui/Global-Flow-Local-Attention.
References in corpus (11)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Spectral Normalization for Generative Adversarial Networks
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Self-Attention Generative Adversarial Networks
- Pose Guided Person Image Generation
- FlowNet: Learning Optical Flow with Convolutional Networks
- FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
- Video-to-Video Synthesis
- Multi-mapping Image-to-Image Translation via Learning Disentanglement
- Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation
Cited by in corpus (5)
- Towards Fine-grained Human Pose Transfer with Detail Replenishing Network
- One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
- Shape Controllable Virtual Try-on for Underwear Models
- Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency
- Sparse to Dense Motion Transfer for Face Image Animation