DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
arXiv:2006.07634
Abstract
As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors. Motivated by the fact that remote visual photoplethysmography (PPG) is made possible by monitoring the minuscule periodic changes of skin color due to blood pumping through the face, we conjecture that normal heartbeat rhythms found in the real face videos will be disrupted or even entirely broken in a DeepFake video, making it a potentially powerful indicator for DeepFake detection. In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes dual-spatial-temporal attention to adapt to dynamically changing face and fake types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have confirmed our conjecture and demonstrated not only the effectiveness, but also the generalization capability of \emph{DeepRhythm} over different datasets by various DeepFakes generation techniques and multifarious challenging degradations.
arXiv comment11 pages, 7 figures; This paper has been accepted to ACM-MM 2020
References in corpus (16)
- MesoNet: a Compact Facial Video Forgery Detection Network
- Progressive Growing of GANs for Improved Quality, Stability, and Variation
- RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation
- Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
- Automatic Face Reenactment
- The Deepfake Detection Challenge (DFDC) Preview Dataset
- Facial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluation
- Use of a Capsule Network to Detect Fake Images and Videos
- Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision
- Detecting GAN-generated Imagery using Color Cues
- Watch out! Motion is Blurring the Vision of Your Deep Neural Networks
- Two-Stream Neural Networks for Tampered Face Detection
- Deferred Neural Rendering: Image Synthesis using Neural Textures
- On the Detection of Digital Face Manipulation
- STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing
- Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack