Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries

Abstract

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy-clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts like JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency domain correlation to analyze the discriminative characteristics between manipulated and non-manipulated regions by incorporating encoder and LSTM network. Finally, decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With predicted mask provided by final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fully Convolutional Networks for Semantic Segmentation
• Conditional Random Fields as Recurrent Neural Networks
• An Evaluation of Popular Copy-Move Forgery Detection Approaches
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

Cited by in corpus (23)

• Media Forensics and DeepFakes: an overview
• MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection
• SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization
• Self-Adversarial Training incorporating Forgery Attention for Image Forgery Localization
• Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics
• Face Forgery Detection by 3D Decomposition
• Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors
• Exposing Fake Images with Forensic Similarity Graphs
• Landmark Tracking in Liver US images Using Cascade Convolutional Neural Networks with Long Short-Term Memory
• CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation
• Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
• Making Images Real Again: A Comprehensive Survey on Deep Image Composition
• Identity-Driven DeepFake Detection
• D-Unet: A Dual-encoder U-Net for Image Splicing Forgery Detection and Localization
• Face X-ray for More General Face Forgery Detection
• Cyber Vaccine for Deepfake Immunity
• Seam Carving Detection and Localization using Two-Stage Deep Neural Networks
• TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization
• FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
• Constrained R-CNN: A general image manipulation detection model
• Self-supervised GAN Detector
• Deepfake Representation with Multilinear Regression
• Holistic Image Manipulation Detection using Pixel Co-occurrence Matrices