iQIYI-VID: A Large Dataset for Multi-modal Person Identification

Abstract

Person identification in the wild is very challenging due to great variation in poses, face quality, clothes, makeup and so on. Traditional research, such as face recognition, person re-identification, and speaker recognition, often focuses on a single modal of information, which is inadequate to handle all the situations in practice. Multi-modal person identification is a more promising way that we can jointly utilize face, head, body, audio features, and so on. In this paper, we introduce iQIYI-VID, the largest video dataset for multi-modal person identification. It is composed of 600K video clips of 5,000 celebrities. These video clips are extracted from 400K hours of online videos of various types, ranging from movies, variety shows, TV series, to news broadcasting. All video clips pass through a careful human annotation process, and the error rate of labels is lower than 0.2\%. We evaluated the state-of-art models of face recognition, person re-identification, and speaker recognition on the iQIYI-VID dataset. Experimental results show that these models are still far from being perfect for the task of person identification in the wild. We proposed a Multi-modal Attention module to fuse multi-modal features that can improve person identification considerably. We have released the dataset online to promote multi-modal person identification research.

References in corpus (18)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• In Defense of the Triplet Loss for Person Re-Identification
• Self-Attention Generative Adversarial Networks
• VoxCeleb2: Deep Speaker Recognition
• VoxCeleb: a large-scale speaker identification dataset
• Improving Person Re-identification by Attribute and Identity Learning
• DeepID3: Face Recognition with Very Deep Neural Networks
• CrowdHuman: A Benchmark for Detecting Human in a Crowd
• DenseBox: Unifying Landmark Localization with End to End Object Detection
• AlignedReID: Surpassing Human-Level Performance in Person Re-Identification
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Deep Speaker: an End-to-End Neural Speaker Embedding System
• Deep Transfer Learning for Person Re-identification
• Face Attention Network: An Effective Face Detector for the Occluded Faces
• Detecting Faces Using Region-based Fully Convolutional Networks
• PyramidBox: A Context-assisted Single Shot Face Detector
• MegaFace: A Million Faces for Recognition at Scale
• Scale-Aware Face Detection

Cited by in corpus (6)

• WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
• Masked Face Recognition Challenge: The WebFace260M Track Report
• When Person Re-identification Meets Changing Clothes
• Long-Term Cloth-Changing Person Re-identification
• Unknown Identity Rejection Loss: Utilizing Unlabeled Data for Face Recognition
• Frame Aggregation and Multi-Modal Fusion Framework for Video-Based Person Recognition