MesoNet: a Compact Facial Video Forgery Detection Network

Abstract

This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. Traditional image forensics techniques are usually not well suited to videos due to the compression that strongly degrades the data. Thus, this paper follows a deep learning approach and presents two networks, both with a low number of layers to focus on the mesoscopic properties of images. We evaluate those fast networks on both an existing dataset and a dataset we have constituted from online videos. The tests demonstrate a very successful detection rate with more than 98% for Deepfake and 95% for Face2Face.

accepted to WIFS 2018

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
• Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks
• Automatic Face Reenactment
• Single Image Super-Resolution with Dilated Convolution based Multi-Scale Information Learning Inception Module

Cited by in corpus (117)

• Media Forensics and DeepFakes: an overview
• The Creation and Detection of Deepfakes: A Survey
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
• Deep Learning for Deepfakes Creation and Detection: A Survey
• Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Identification of Deep Network Generated Images Using Disparities in Color Components
• ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
• Unmasking DeepFakes with simple Features
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Sharp Multiple Instance Learning for DeepFake Video Detection
• Deepfake Video Detection Using Convolutional Vision Transformer
• Use of a Capsule Network to Detect Fake Images and Videos
• Deep Convolutional Pooling Transformer for Deepfake Detection
• Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
• Face Forgery Detection by 3D Decomposition
• Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors
• Deepfake Detection: A Comprehensive Survey from the Reliability Perspective
• Social bot detection in the age of ChatGPT: Challenges and opportunities
• One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework
• A Review of Deep Learning-based Approaches for Deepfake Content Detection
• Wavelet-Packets for Deepfake Image Analysis and Detection
• DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
• Deepfake Detection using Spatiotemporal Convolutional Networks
• DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation
• Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking
• DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
• FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point Defects
• CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation
• Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues
• On the Detection of Digital Face Manipulation
• Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
• Preliminary Forensics Analysis of DeepFake Images
• Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
• Learning to Discover Forgery Cues for Face Forgery Detection
• Jekyll: Attacking Medical Image Diagnostics using Deep Generative Models
• Attentive Semantic Exploring for Manipulated Face Detection
• Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection
• DeepFake Detection Based on the Discrepancy Between the Face and its Context
• Towards a Robust Deep Neural Network in Texts: A Survey
• LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks
• Predicting Video with VQVAE
• Generalizing Face Forgery Detection with High-frequency Features
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
• FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
• Hiding Faces in Plain Sight: Disrupting AI Face Synthesis with Adversarial Perturbations
• Identity-Driven DeepFake Detection
• Face X-ray for More General Face Forgery Detection
• Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection
• Responsible Disclosure of Generative Models Using Scalable Fingerprinting
• AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection
• Exposing Deep-faked Videos by Anomalous Co-motion Pattern Detection
• Cyber Vaccine for Deepfake Immunity
• Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses
• Am I a Real or Fake Celebrity? Measuring Commercial Face Recognition Web APIs under Deepfake Impersonation Attack
• AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection
• Multi-attentional Deepfake Detection
• Adversarially Robust Deepfake Detection via Adversarial Feature Similarity Learning
• DefakeHop: A Light-Weight High-Performance Deepfake Detector
• Adversarially robust deepfake media detection using fused convolutional neural network predictions
• What makes fake images detectable? Understanding properties that generalize
• Detecting Deepfake-Forged Contents with Separable Convolutional Neural Network and Image Segmentation
• Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder
• Transformation-based Adversarial Video Prediction on Large-Scale Data
• Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues
• From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users
• Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images
• Investigating the Impact of Pre-processing and Prediction Aggregation on the DeepFake Detection Task
• Combating Fake News with Interpretable News Feed Algorithms
• Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks
• On the generalization of GAN image forensics
• How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals
• Identifying Invariant Texture Violation for Robust Deepfake Detection
• MMSys'21 Grand Challenge on Detecting Cheapfakes
• Mining Generalized Features for Detecting AI-Manipulated Fake Faces
• A Person Re-identification Data Augmentation Method with Adversarial Defense Effect
• Zooming into Face Forensics: A Pixel-level Analysis
• FastStamp: Accelerating Neural Steganography and Digital Watermarking of Images on FPGAs
• Fake face detection via adaptive manipulation traces extraction network
• Deep Fake Detection: Survey of Facial Manipulation Detection Solutions
• Towards Generalizable and Robust Face Manipulation Detection via Bag-of-local-feature
• FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
• AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Deepfake Detection of Frontal Face Videos
• An Examination of Fairness of AI Models for Deepfake Detection
• 3D Face Anti-spoofing with Factorized Bilinear Coding
• FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
• Towards General Deepfake Detection with Dynamic Curriculum
• Can deepfakes be created by novice users?
• DeepFake MNIST+: A DeepFake Facial Animation Dataset
• Detection of Deepfake Videos Using Long Distance Attention
• StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting
• Video Transformer for Deepfake Detection with Incremental Learning
• Dodging DeepFake Detection via Implicit Spatial-Domain Notch Filtering
• The eyes know it: FakeET -- An Eye-tracking Database to Understand Deepfake Perception
• Automated Deepfake Detection
• Designing Indicators to Combat Fake Media
• A Method for Identifying Origin of Digital Images Using a Convolution Neural Network
• Towards Exploring Fairness in Visual Transformer based Natural and GAN Image Detection Systems
• Contrastive Learning of Global-Local Video Representations
• Deep Detection for Face Manipulation
• Deepfake Videos in the Wild: Analysis and Detection
• Perceptual Indistinguishability-Net (PI-Net): Facial Image Obfuscation with Manipulable Semantics
• What's wrong with this video? Comparing Explainers for Deepfake Detection
• Exposing Deepfake with Pixel-wise AR and PPG Correlation from Faint Signals
• Impact of Benign Modifications on Discriminative Performance of Deepfake Detectors
• Seeing Through the Blur: Unlocking Defocus Maps for Deepfake Detection
• Learning to identify image manipulations in scientific publications
• Metric Learning for Anti-Compression Facial Forgery Detection
• Finding Facial Forgery Artifacts with Parts-Based Detectors
• DeepFake-o-meter: An Open Platform for DeepFake Detection
• Exploring Adversarial Fake Images on Face Manifold
• DeepFake Detection with Inconsistent Head Poses: Reproducibility and Analysis
• Deepfake Representation with Multilinear Regression
• Landmark Breaker: Obstructing DeepFake By Disturbing Landmark Extraction
• Combating Falsification of Speech Videos with Live Optical Signatures (Extended Version)