Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship
arXiv:2007.14509 · doi:10.1109/TMM.2021.3103074
Abstract
Kinship, a soft biometric detectable in media, is fundamental for a myriad of use-cases. Despite the difficulty of detecting kinship, annual data challenges using still-images have consistently improved performances and attracted new researchers. Now, systems reach performance levels unforeseeable a decade ago, closing in on performances acceptable to deploy in practice. Like other biometric tasks, we expect systems can receive help from other modalities. We hypothesize that adding modalities to FIW, which has only still-images, will improve performance. Thus, to narrow the gap between research and reality and enhance the power of kinship recognition systems, we extend FIW with multimedia (MM) data (i.e., video, audio, and text captions). Specifically, we introduce the first publicly available multi-task MM kinship dataset. To build FIW MM, we developed machinery to automatically collect, annotate, and prepare the data, requiring minimal human input and no financial cost. The proposed MM corpus allows the problem statements to be more realistic template-based protocols. We show significant improvements in all benchmarks with the added modalities. The results highlight edge cases to inspire future research with different areas of improvement. FIW MM supplies the data needed to increase the potential of automated systems to detect kinship in MM. It also allows experts from diverse fields to collaborate in novel ways.
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
- Prototypical Networks for Few-shot Learning
- VoxCeleb2: Deep Speaker Recognition
- VoxCeleb: a large-scale speaker identification dataset
- Lip Reading Sentences in the Wild
- Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation
- NormFace: L2 Hypersphere Embedding for Face Verification
- In defence of metric learning for speaker recognition
- Self-supervised learning of a facial attribute embedding from video
- Families in the Wild (FIW): Large-Scale Kinship Image Database and Benchmarks
- A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
- Recognizing Families In the Wild: White Paper for the 4th Edition Data Challenge
- Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making
- What Will Your Child Look Like? DNA-Net: Age and Gender Aware Kin Face Synthesizer