Media Forensics and DeepFakes: an overview
arXiv:2001.06564 · doi:10.1109/JSTSP.2020.3002101
Abstract
With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.
References in corpus (42)
- Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Generative Adversarial Text to Image Synthesis
- MesoNet: a Compact Facial Video Forgery Detection Network
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- YouTube-8M: A Large-Scale Video Classification Benchmark
- An Evaluation of Popular Copy-Move Forgery Detection Approaches
- Exposing DeepFake Videos By Detecting Face Warping Artifacts
- FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
- DeepFakes: a New Threat to Face Recognition? Assessment and Detection
- Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
- Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)
- FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
- Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
- Semantic Image Synthesis with Spatially-Adaptive Normalization
- Identification of Deep Network Generated Images Using Disparities in Color Components
- Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks
- ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
- The Deepfake Detection Challenge (DFDC) Preview Dataset
- An Introduction to Image Synthesis with Generative Adversarial Nets
- Video-to-Video Synthesis
- Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
- A Counter-Forensic Method for CNN-Based Camera Model Identification
- You said that?
- Exposing Fake Images with Forensic Similarity Graphs
- DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
- Adversarial Scene Editing: Automatic Object Removal from Weak Supervision
- RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes
- On the Detection of Digital Face Manipulation
- Detecting and Simulating Artifacts in GAN Fake Images
- We Need No Pixels: Video Manipulation Detection Using Stream Descriptors
- Detecting Photoshopped Faces by Scripting Photoshop
- Hiding Faces in Plain Sight: Disrupting AI Face Synthesis with Adversarial Perturbations
- Face X-ray for More General Face Forgery Detection
- Exposing GAN-synthesized Faces Using Landmark Locations
- GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection
- Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder
- Deep Matching and Validation Network -- An End-to-End Solution to Constrained Image Splicing Localization and Detection
- Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks
- Generate, Segment and Refine: Towards Generic Manipulation Segmentation
- On the generalization of GAN image forensics
- Zooming into Face Forensics: A Pixel-level Analysis
- Analysis of adversarial attacks against CNN-based image forgery detectors
Cited by in corpus (26)
- Deep Learning for Deepfakes Creation and Detection: A Survey
- MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection
- Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds
- Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization
- Fighting Deepfake by Exposing the Convolutional Traces on Images
- Fighting deepfakes by detecting GAN DCT anomalies
- Self-Adversarial Training incorporating Forgery Attention for Image Forgery Localization
- Face Forgery Detection by 3D Decomposition
- Face Generation and Editing with StyleGAN: A Survey
- More Real than Real: A Study on Human Visual Perception of Synthetic Faces
- Exposing Fake Images with Forensic Similarity Graphs
- DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation
- Deepfake: Definitions, Performance Metrics and Standards, Datasets and Benchmarks, and a Meta-Review
- The DeepFake Detection Challenge (DFDC) Dataset
- Shrinking the Semantic Gap: Spatial Pooling of Local Moment Invariants for Copy-Move Forgery Detection
- Preliminary Forensics Analysis of DeepFake Images
- Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
- A Principled Design of Image Representation: Towards Forensic Tasks
- Differential Anomaly Detection for Facial Images
- Video Source Characterization Using Encoding and Encapsulation Characteristics
- GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection
- Investigating the Impact of Pre-processing and Prediction Aggregation on the DeepFake Detection Task
- Impact of Facial Tattoos and Paintings on Face Recognition Systems
- Representing Noisy Image Without Denoising
- Deepfake Style Transfer Mixture: a First Forensic Ballistics Study on Synthetic Images
- Training Data Improvement for Image Forgery Detection using Comprint