SimSwap: An Efficient Framework For High Fidelity Face Swapping

Abstract

We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity or fail to preserve attributes like facial expression and gaze direction, our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face. We overcome the above defects in the following two ways. First, we present the ID Injection Module (IIM) which transfers the identity information of the source face into the target face at feature level. By using this module, we extend the architecture of an identity-specific face swapping algorithm to a framework for arbitrary face swapping. Second, we propose the Weak Feature Matching Loss which efficiently helps our framework to preserve the facial attributes in an implicit way. Extensive experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods. The code is already available on github: https://github.com/neuralchen/SimSwap.

Accepted by ACMMM 2020

References in corpus (2)

• RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces
• FSNet: An Identity-Aware Generative Model for Image-based Face Swapping

Cited by in corpus (14)

• Deepfake Detection: A Comprehensive Survey from the Reliability Perspective
• LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks
• Deep Learning Technique for Human Parsing: A Survey and Outlook
• Cyber Vaccine for Deepfake Immunity
• FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks
• FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing
• AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Deepfake Detection of Frontal Face Videos
• Steganography Beyond Space-Time with Chain of Multimodal AI
• Parametric Reshaping of Portraits in Videos
• MFIM: Megapixel Facial Identity Manipulation
• Introducing Explicit Gaze Constraints to Face Swapping
• FaceTracer: Unveiling Source Identities from Swapped Face Images and Videos for Fraud Prevention
• Além do Desempenho: Um Estudo da Confiabilidade de Detectores de Deepfakes
• Transferable Attack against Face Swapping in an Extended Space