Image-Text Out-Of-Context Detection Using Synthetic Multimodal Misinformation

Abstract

Misinformation has become a major challenge in the era of increasing digital information, requiring the development of effective detection methods. We have investigated a novel approach to Out-Of-Context detection (OOCD) that uses synthetic data generation. We created a dataset specifically designed for OOCD and developed an efficient detector for accurate classification. Our experimental findings validate the use of synthetic data generation and demonstrate its efficacy in addressing the data limitations associated with OOCD. The dataset and detector should serve as valuable resources for future research and the development of robust misinformation detection systems.

arXiv comment8 pages, 2 figures, conference

References in corpus (16)

• Denoising Diffusion Probabilistic Models
• Generative Adversarial Networks: An Overview
• Neural Discrete Representation Learning
• Diffusion Models Beat GANs on Image Synthesis
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Flamingo: a Visual Language Model for Few-Shot Learning
• VisualBERT: A Simple and Performant Baseline for Vision and Language
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Classifier-Free Diffusion Guidance
• Reproducible scaling laws for contrastive language-image learning
• CogView: Mastering Text-to-Image Generation via Transformers
• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
• Deep Multimodal Image-Repurposing Detection
• Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text
• PromptCap: Prompt-Guided Task-Aware Image Captioning
• Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Model