Deepfake Video Detection Using Convolutional Vision Transformer

Abstract

The rapid advancement of deep learning models that can generate and synthesis hyper-realistic videos known as Deepfakes and their ease of access to the general public have raised concern from all concerned bodies to their possible malicious intent use. Deep learning techniques can now generate faces, swap faces between two subjects in a video, alter facial expressions, change gender, and alter facial features, to list a few. These powerful video manipulation methods have potential use in many fields. However, they also pose a looming threat to everyone if used for harmful purposes such as identity theft, phishing, and scam. In this work, we propose a Convolutional Vision Transformer for the detection of Deepfakes. The Convolutional Vision Transformer has two components: Convolutional Neural Network (CNN) and Vision Transformer (ViT). The CNN extracts learnable features while the ViT takes in the learned features as input and categorizes them using an attention mechanism. We trained our model on the DeepFake Detection Challenge Dataset (DFDC) and have achieved 91.5 percent accuracy, an AUC value of 0.91, and a loss value of 0.32. Our contribution is that we have added a CNN module to the ViT architecture and have achieved a competitive result on the DFDC dataset.

9 pages, 6 figures

References in corpus (13)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• MesoNet: a Compact Facial Video Forgery Detection Network
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
• DeepFakes: a New Threat to Face Recognition? Assessment and Detection
• How Generative Adversarial Networks and Their Variants Work: An Overview
• BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs
• In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking
• An Introduction to Image Synthesis with Generative Adversarial Nets
• RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces
• Adversarial Audio Synthesis
• Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network
• Investigating the Impact of Pre-processing and Prediction Aggregation on the DeepFake Detection Task

Cited by in corpus (7)

• Combining EfficientNet and Vision Transformers for Video Deepfake Detection
• A Review of Deep Learning-based Approaches for Deepfake Content Detection
• FFR_FD: Effective and Fast Detection of DeepFakes Based on Feature Point Defects
• Cross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
• Enhancing Deepfake Detection using SE Block Attention with CNN
• Detection of Deepfake Videos Using Long Distance Attention
• Towards Solving the DeepFake Problem : An Analysis on Improving DeepFake Detection using Dynamic Face Augmentation