CLIP-ReIdent: Contrastive Training for Player Re-Identification

Abstract

Sports analytics benefits from recent advances in machine learning providing a competitive advantage for teams or individuals. One important task in this context is the performance measurement of individual players to provide reports and log files for subsequent analysis. During sport events like basketball, this involves the re-identification of players during a match either from multiple camera viewpoints or from a single camera viewpoint at different times. In this work, we investigate whether it is possible to transfer the out-standing zero-shot performance of pre-trained CLIP models to the domain of player re-identification. For this purpose we reformulate the contrastive language-to-image pre-training approach from CLIP to a contrastive image-to-image training approach using the InfoNCE loss as training objective. Unlike previous work, our approach is entirely class-agnostic and benefits from large-scale pre-training. With a fine-tuned CLIP ViT-L/14 model we achieve 98.44 % mAP on the MMSports 2022 Player Re-Identification challenge. Furthermore we show that the CLIP Vision Transformers have already strong OCR capabilities to identify useful player features like shirt numbers in a zero-shot manner without any fine-tuning on the dataset. By applying the Score-CAM algorithm we visualise the most important image regions that our fine-tuned model identifies when calculating the similarity score between two images of a player.

References in corpus (11)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Decoupled Weight Decay Regularization
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Learning Transferable Visual Models From Natural Language Supervision
• Contrastive Learning of Medical Visual Representations from Paired Images and Text
• SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos
• Learning to Disentangle Scenes for Person Re-identification
• DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations
• Eliminate Deviation with Deviation for Data Augmentation and a General Multi-modal Data Learning Method
• Sports Re-ID: Improving Re-Identification Of Players In Broadcast Videos Of Team Sports

Cited by in corpus (2)

• SoccerNet 2023 Challenges Results
• Runner re-identification from single-view running video in the open-world setting