Deep Learning for Deepfakes Creation and Detection: A Survey

Abstract

Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is deepfake. Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity of digital visual media is therefore indispensable. This paper presents a survey of algorithms used to create deepfakes and, more importantly, methods proposed to detect deepfakes in the literature to date. We present extensive discussions on challenges, research trends and directions related to deepfake technologies. By reviewing the background of deepfakes and state-of-the-art deepfake detection methods, this study provides a comprehensive overview of deepfake techniques and facilitates the development of new and more robust methods to deal with the increasingly challenging deepfakes.

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Self-Attention Generative Adversarial Networks
• MesoNet: a Compact Facial Video Forgery Detection Network
• A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities
• Media Forensics and DeepFakes: an overview
• The Creation and Detection of Deepfakes: A Survey
• Exposing DeepFake Videos By Detecting Face Warping Artifacts
• Detection and Resolution of Rumours in Social Media: A Survey
• FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
• DeepFakes: a New Threat to Face Recognition? Assessment and Detection
• FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
• Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
• ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
• Unsupervised speech representation learning using WaveNet autoencoders
• FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
• Fighting Deepfake by Exposing the Convolutional Traces on Images
• Fighting deepfakes by detecting GAN DCT anomalies
• Limits of Deepfake Detection: A Robust Estimation Viewpoint

Cited by in corpus (10)

• Face Generation and Editing with StyleGAN: A Survey
• A Review of Deep Learning-based Approaches for Deepfake Content Detection
• Deepfake: Definitions, Performance Metrics and Standards, Datasets and Benchmarks, and a Meta-Review
• BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis
• The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hub
• Social Media Authentication and Combating Deepfakes using Semi-fragile Invisible Image Watermarking
• SpecXNet: A Dual-Domain Convolutional Network for Robust Deepfake Detection
• CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features
• Phantom: Untargeted Poisoning Attacks on Semi-Supervised Learning (Full Version)
• "Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detection