What Matters for Ad-hoc Video Search? A Large-scale Evaluation on TRECVID
arXiv:2109.01774
Abstract
For quantifying progress in Ad-hoc Video Search (AVS), the annual TRECVID AVS task is an important international evaluation. Solutions submitted by the task participants vary in terms of their choices of cross-modal matching models, visual features and training data. As such, what one may conclude from the evaluation is at a high level that is insufficient to reveal the influence of the individual components. In order to bridge the gap between the current solution-level comparison and the desired component-wise comparison, we propose in this paper a large-scale and systematic evaluation on TRECVID. By selected combinations of state-of-the-art matching models, visual features and (pre-)training data, we construct a set of 25 different solutions and evaluate them on the TRECVID AVS tasks 2016--2020. The presented evaluation helps answer the key question of what matters for AVS. The resultant observations and learned lessons are also instructive for developing novel AVS solutions.
arXiv commentAccepted by ViRal'21@ICCV 2021
References in corpus (5)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Learning Transferable Visual Models From Natural Language Supervision
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
- Dual Encoding for Video Retrieval by Text
- Feature Re-Learning with Data Augmentation for Video Relevance Prediction