Deepfake Detection Scheme Based on Vision Transformer and Distillation

Abstract

Deepfake is the manipulated video made with a generative deep learning technique such as Generative Adversarial Networks (GANs) or Auto Encoder that anyone can utilize. Recently, with the increase of Deepfake videos, some classifiers consisting of the convolutional neural network that can distinguish fake videos as well as deepfake datasets have been actively created. However, the previous studies based on the CNN structure have the problem of not only overfitting, but also considerable misjudging fake video as real ones. In this paper, we propose a Vision Transformer model with distillation methodology for detecting fake videos. We design that a CNN features and patch-based positioning model learns to interact with all positions to find the artifact region for solving false negative problem. Through comparative analysis on Deepfake Detection (DFDC) Dataset, we verify that the proposed scheme with patch embedding as input outperforms the state-of-the-art using the combined CNN features. Without ensemble technique, our model obtains 0.978 of AUC and 91.9 of f1 score, while previous SOTA model yields 0.972 of AUC and 90.6 of f1 score on the same condition.

7 pages, 5 figures

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Language Models are Few-Shot Learners
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• DeepFakes: a New Threat to Face Recognition? Assessment and Detection
• The Deepfake Detection Challenge (DFDC) Preview Dataset
• Use of a Capsule Network to Detect Fake Images and Videos
• Deepfake Detection using Spatiotemporal Convolutional Networks
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• The DeepFake Detection Challenge (DFDC) Dataset

Cited by in corpus (5)

• Combining EfficientNet and Vision Transformers for Video Deepfake Detection
• Video Transformers: A Survey
• Cross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
• Detection of Deepfake Videos Using Long Distance Attention
• Towards Solving the DeepFake Problem : An Analysis on Improving DeepFake Detection using Dynamic Face Augmentation