DeepFake Detection Based on the Discrepancy Between the Face and its Context
arXiv:2008.12262
Abstract
We propose a method for detecting face swapping and other identity manipulations in single images. Face swapping methods, such as DeepFake, manipulate the face region, aiming to adjust the face to the appearance of its context, while leaving the context unchanged. We show that this modus operandi produces discrepancies between the two regions. These discrepancies offer exploitable telltale signs of manipulation. Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context (e.g., hair, ears, neck). We describe a method which uses the recognition signals from our two networks to detect such discrepancies, providing a complementary detection signal that improves conventional real vs. fake classifiers commonly used for detecting fake images. Our method achieves state of the art results on the FaceForensics++, Celeb-DF-v2, and DFDC benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.
References in corpus (14)
- Adam: A Method for Stochastic Optimization
- MesoNet: a Compact Facial Video Forgery Detection Network
- Exposing DeepFake Videos By Detecting Face Warping Artifacts
- FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
- In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking
- The Deepfake Detection Challenge (DFDC) Preview Dataset
- Use of a Capsule Network to Detect Fake Images and Videos
- Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
- Deferred Neural Rendering: Image Synthesis using Neural Textures
- On the Detection of Digital Face Manipulation
- Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
- Detecting Photoshopped Faces by Scripting Photoshop
- Complement Face Forensic Detection and Localization with FacialLandmarks
- Triple consistency loss for pairing distributions in GAN-based face synthesis