Media Forensics and DeepFakes: an overview

Abstract

With the rapid progress of recent years, techniques that generate and manipulate multimedia content can now guarantee a very advanced level of realism. The boundary between real and synthetic media has become very thin. On the one hand, this opens the door to a series of exciting applications in different fields such as creative arts, advertising, film production, video games. On the other hand, it poses enormous security threats. Software packages freely available on the web allow any individual, without special skills, to create very realistic fake images and videos. So-called deepfakes can be used to manipulate public opinion during elections, commit fraud, discredit or blackmail people. Potential abuses are limited only by human imagination. Therefore, there is an urgent need for automated tools capable of detecting false multimedia content and avoiding the spread of dangerous false information. This review paper aims to present an analysis of the methods for visual media integrity verification, that is, the detection of manipulated images and videos. Special emphasis will be placed on the emerging phenomenon of deepfakes and, from the point of view of the forensic analyst, on modern data-driven forensic methods. The analysis will help to highlight the limits of current forensic tools, the most relevant issues, the upcoming challenges, and suggest future directions for research.

References in corpus (14)

• YouTube-8M: A Large-Scale Video Classification Benchmark
• An Evaluation of Popular Copy-Move Forgery Detection Approaches
• DeepFakes: a New Threat to Face Recognition? Assessment and Detection
• Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
• Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
• Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks
• The Deepfake Detection Challenge (DFDC) Preview Dataset
• DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
• We Need No Pixels: Video Manipulation Detection Using Stream Descriptors
• Hiding Faces in Plain Sight: Disrupting AI Face Synthesis with Adversarial Perturbations
• Exposing GAN-synthesized Faces Using Landmark Locations
• Deep Matching and Validation Network -- An End-to-End Solution to Constrained Image Splicing Localization and Detection
• Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks
• Zooming into Face Forensics: A Pixel-level Analysis

Cited by in corpus (23)

• MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection
• Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds
• Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization
• Fighting Deepfake by Exposing the Convolutional Traces on Images
• Fighting deepfakes by detecting GAN DCT anomalies
• Self-Adversarial Training incorporating Forgery Attention for Image Forgery Localization
• Face Forgery Detection by 3D Decomposition
• Face Generation and Editing with StyleGAN: A Survey
• More Real than Real: A Study on Human Visual Perception of Synthetic Faces
• DeepFakesON-Phys: DeepFakes Detection based on Heart Rate Estimation
• Deepfake: Definitions, Performance Metrics and Standards, Datasets and Benchmarks, and a Meta-Review
• The DeepFake Detection Challenge (DFDC) Dataset
• Shrinking the Semantic Gap: Spatial Pooling of Local Moment Invariants for Copy-Move Forgery Detection
• Preliminary Forensics Analysis of DeepFake Images
• Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
• A Principled Design of Image Representation: Towards Forensic Tasks
• Differential Anomaly Detection for Facial Images
• Video Source Characterization Using Encoding and Encapsulation Characteristics
• Investigating the Impact of Pre-processing and Prediction Aggregation on the DeepFake Detection Task
• Impact of Facial Tattoos and Paintings on Face Recognition Systems
• Deepfake Style Transfer Mixture: a First Forensic Ballistics Study on Synthetic Images
• Representing Noisy Image Without Denoising
• Training Data Improvement for Image Forgery Detection using Comprint