RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces

Abstract

In this paper, we present an integrated system for automatically generating and editing face images through face swapping, attribute-based editing, and random face parts synthesis. The proposed system is based on a deep neural network that variationally learns the face and hair regions with large-scale face image datasets. Different from conventional variational methods, the proposed network represents the latent spaces individually for faces and hairs. We refer to the proposed network as region-separative generative adversarial network (RSGAN). The proposed network independently handles face and hair appearances in the latent spaces, and then, face swapping is achieved by replacing the latent-space representations of the faces, and reconstruct the entire face image with them. This approach in the latent space robustly performs face swapping even for images which the previous methods result in failure due to inappropriate fitting or the 3D morphable models. In addition, the proposed system can further edit face-swapped images with the same network by manipulating visual attributes or by composing them with randomly generated face or hair parts.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• TensorFlow: A system for large-scale machine learning
• Semi-Supervised Learning with Deep Generative Models
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Neural Photo Editing with Introspective Adversarial Networks
• LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
• Generating Images Part by Part with Composite Generative Adversarial Networks
• High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks

Cited by in corpus (11)

• SimSwap: An Efficient Framework For High Fidelity Face Swapping
• Deepfake Video Detection Using Convolutional Vision Transformer
• Deep Convolutional Pooling Transformer for Deepfake Detection
• Deepfake Detection: A Comprehensive Survey from the Reliability Perspective
• FSNet: An Identity-Aware Generative Model for Image-based Face Swapping
• Identity-Driven DeepFake Detection
• Multi-attentional Deepfake Detection
• Deepfake Videos in the Wild: Analysis and Detection
• Metric Learning for Anti-Compression Facial Forgery Detection
• FaceTracer: Unveiling Source Identities from Swapped Face Images and Videos for Fraud Prevention
• Causal Representation Learning for Context-Aware Face Transfer