Face X-ray for More General Face Forgery Detection

Abstract

In this paper we propose a novel image representation called face X-ray for detecting forgery in face images. The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources. It does so by showing the blending boundary for a forged image and the absence of blending for a real image. We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image. For this reason, face X-ray provides an effective way for detecting forgery generated by most existing face manipulation algorithms. Face X-ray is general in the sense that it only assumes the existence of a blending step and does not rely on any knowledge of the artifacts associated with a specific face manipulation technique. Indeed, the algorithm for computing face X-ray can be trained without fake images generated by any of the state-of-the-art face manipulation methods. Extensive experiments show that face X-ray remains effective when applied to forgery generated by unseen face manipulation techniques, while most existing face forgery detection or deepfake detection algorithms experience a significant performance drop.

Accepted to CVPR 2020 (Oral)

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• MesoNet: a Compact Facial Video Forgery Detection Network
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• High-Resolution Representations for Labeling Pixels and Regions
• Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
• Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)
• FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
• Identification of Deep Network Generated Images Using Disparities in Color Components
• ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
• Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
• Deep High-Resolution Representation Learning for Human Pose Estimation
• Deferred Neural Rendering: Image Synthesis using Neural Textures
• On the Detection of Digital Face Manipulation
• Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
• FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
• Complement Face Forensic Detection and Localization with FacialLandmarks
• Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder
• Digital Image Forensics vs. Image Composition: An Indirect Arms Race
• Swapped Face Detection using Deep Learning and Subjective Assessment

Cited by in corpus (5)

• Media Forensics and DeepFakes: an overview
• FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
• Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues
• Fake Generated Painting Detection via Frequency Analysis
• Defending against GAN-based Deepfake Attacks via Transformation-aware Adversarial Faces