Non-local NetVLAD Encoding for Video Classification

Abstract

This paper describes our solution for the 2${}^{\text{nd}}$ YouTube-8M video understanding challenge organized by Google AI. Unlike the video recognition benchmarks, such as Kinetics and Moments, the YouTube-8M challenge provides pre-extracted visual and audio features instead of raw videos. In this challenge, the submitted model is restricted to 1GB, which encourages participants focus on constructing one powerful single model rather than incorporating of the results from a bunch of models. Our system fuses six different sub-models into one single computational graph, which are categorized into three families. More specifically, the most effective family is the model with non-local operations following the NetVLAD encoding. The other two family models are Soft-BoF and GRU, respectively. In order to further boost single models performance, the model parameters of different checkpoints are averaged. Experimental results demonstrate that our proposed system can effectively perform the video classification task, achieving 0.88763 on the public test set and 0.88704 on the private set in terms of GAP@20, respectively. We finally ranked at the fourth place in the YouTube-8M video understanding challenge.

arXiv commentECCV2018 workshop on YouTube-8M Large-Scale Video Understanding

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• The Kinetics Human Action Video Dataset
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
• Learnable pooling with Context Gating for video classification
• Averaging Weights Leads to Wider Optima and Better Generalization
• Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding
• Reconstruction Network for Video Captioning
• Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning
• The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
• Aggregating Frame-level Features for Large-Scale Video Classification
• Qiniu Submission to ActivityNet Challenge 2018

Cited by in corpus (5)

• Long-Term Feature Banks for Detailed Video Understanding
• Combinational Class Activation Maps for Weakly Supervised Object Localization
• Efficient Video Classification Using Fewer Frames
• Hallucinating Optical Flow Features for Video Classification
• MOD: A Deep Mixture Model with Online Knowledge Distillation for Large Scale Video Temporal Concept Localization