OFAR: A Multimodal Evidence Retrieval Framework for Illegal Live-streaming Identification

Abstract

Illegal live-streaming identification, which aims to help live-streaming platforms immediately recognize the illegal behaviors in the live-streaming, such as selling precious and endangered animals, plays a crucial role in purifying the network environment. Traditionally, the live-streaming platform needs to employ some professionals to manually identify the potential illegal live-streaming. Specifically, the professional needs to search for related evidence from a large-scale knowledge database for evaluating whether a given live-streaming clip contains illegal behavior, which is time-consuming and laborious. To address this issue, in this work, we propose a multimodal evidence retrieval system, named OFAR, to facilitate the illegal live-streaming identification. OFAR consists of three modules: Query Encoder, Document Encoder, and MaxSim-based Contrastive Late Intersection. Both query encoder and document encoder are implemented with the advanced OFA encoder, which is pretrained on a large-scale multimodal dataset. In the last module, we introduce contrastive learning on the basis of the MaxiSim-based late intersection, to enhance the model's ability of query-document matching. The proposed framework achieves significant improvement on our industrial dataset TaoLive, demonstrating the advances of our scheme.

References in corpus (7)

• A Simple Framework for Contrastive Learning of Visual Representations
• ERNIE: Enhanced Representation through Knowledge Integration
• Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)
• OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
• CEDR: Contextualized Embeddings for Document Ranking
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue