BERT for Large-scale Video Segment Classification with Test-time Augmentation
arXiv:1912.01127
Abstract
This paper presents our approach to the third YouTube-8M video understanding competition that challenges par-ticipants to localize video-level labels at scale to the pre-cise time in the video where the label actually occurs. Ourmodel is an ensemble of frame-level models such as GatedNetVLAD and NeXtVLAD and various BERT models withtest-time augmentation. We explore multiple ways to ag-gregate BERT outputs as video representation and variousways to combine visual and audio information. We proposetest-time augmentation as shifting video frames to one leftor right unit, which adds variety to the predictions and em-pirically shows improvement in evaluation metrics. We firstpre-train the model on the 4M training video-level data, andthen fine-tune the model on 237K annotated video segment-level data. We achieve MAP@100K 0.7871 on private test-ing video segment data, which is ranked 9th over 283 teams.
ICCV 2019 YouTube8M workshop
References in corpus (9)
- Distilling the Knowledge in a Neural Network
- Practical Bayesian Optimization of Machine Learning Algorithms
- The Effectiveness of Data Augmentation in Image Classification using Deep Learning
- YouTube-8M: A Large-Scale Video Classification Benchmark
- Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding
- Characterizing Spatiotemporal Transcriptome of Human Brain via Low Rank Tensor Decomposition
- The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
- Deep Learning Methods for Efficient Large Scale Video Labeling
- Aggregating Frame-level Features for Large-Scale Video Classification