CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation

Abstract

Over the last few decades, artificial intelligence research has made tremendous strides, but it still heavily relies on fixed datasets in stationary environments. Continual learning is a growing field of research that examines how AI systems can learn sequentially from a continuous stream of linked data in the same way that biological systems do. Simultaneously, fake media such as deepfakes and synthetic face images have emerged as significant to current multimedia technologies. Recently, numerous method has been proposed which can detect deepfakes with high accuracy. However, they suffer significantly due to their reliance on fixed datasets in limited evaluation settings. Therefore, in this work, we apply continuous learning to neural networks' learning dynamics, emphasizing its potential to increase data efficiency significantly. We propose Continual Representation using Distillation (CoReD) method that employs the concept of Continual Learning (CL), Representation Learning (RL), and Knowledge Distillation (KD). We design CoReD to perform sequential domain adaptation tasks on new deepfake and GAN-generated synthetic face datasets, while effectively minimizing the catastrophic forgetting in a teacher-student model setting. Our extensive experimental results demonstrate that our method is efficient at domain adaptation to detect low-quality deepfakes videos and GAN-generated images from several datasets, outperforming the-state-of-art baseline methods.

13 pages, 7 Figures, 13 Tables, Accepted for publication in the 29th ACM International Conference on Multimedia (ACMMM '21)

References in corpus (10)

• Overcoming catastrophic forgetting in neural networks
• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• MesoNet: a Compact Facial Video Forgery Detection Network
• The Creation and Detection of Deepfakes: A Survey
• Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
• Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)
• Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
• Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
• One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework
• Lower Dimensional Kernels for Video Discriminators

Cited by in corpus (7)

• Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors
• Deepfake in the Metaverse: Security Implications for Virtual Gaming, Meetings, and Offices
• Why Do Facial Deepfake Detectors Fail?
• Continuous fake media detection: adapting deepfake detectors to new generative techniques
• UGAD: Universal Generative AI Detector utilizing Frequency Fingerprints
• From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users
• Seeing Through the Blur: Unlocking Defocus Maps for Deepfake Detection