A Discriminative CNN Video Representation for Event Detection

Abstract

In this paper, we propose a discriminative video representation for event detection over a large scale video dataset when only limited hardware resources are available. The focus of this paper is to effectively leverage deep Convolutional Neural Networks (CNNs) to advance event detection, where only frame level static descriptors can be extracted by the existing CNN toolkit. This paper makes two contributions to the inference of CNN video representation. First, while average pooling and max pooling have long been the standard approaches to aggregating frame level static features, we show that performance can be significantly improved by taking advantage of an appropriate encoding method. Second, we propose using a set of latent concept descriptors as the frame descriptor, which enriches visual information while keeping it computationally affordable. The integration of the two contributions results in a new state-of-the-art performance in event detection over the largest video datasets. Compared to improved Dense Trajectories, which has been recognized as the best video representation for event detection, our new representation improves the Mean Average Precision (mAP) from 27.6% to 36.8% for the TRECVID MEDTest 14 dataset and from 34.0% to 44.6% for the TRECVID MEDTest 13 dataset. This work is the core part of the winning solution of our CMU-Informedia team in TRECVID MED 2014 competition.

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice

Cited by in corpus (19)

• Exploiting Local Features from Deep Networks for Image Retrieval
• Exploiting Image-trained CNN Architectures for Unconstrained Video Classification
• CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
• Objects2action: Classifying and localizing actions without any video example
• Transfer Learning for Video Recognition with Scarce Training Data for Deep Convolutional Neural Network
• SA-CNN: Dynamic Scene Classification using Convolutional Neural Networks
• Spatio-temporal Human Action Localisation and Instance Segmentation in Temporally Untrimmed Videos
• UC Merced Submission to the ActivityNet Challenge 2016
• ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks
• Deep Quantization: Encoding Convolutional Activations with Deep Generative Model
• Improving Human Activity Recognition Through Ranking and Re-ranking
• ActionVLAD: Learning spatio-temporal aggregation for action classification
• Learning Temporal Embeddings for Complex Video Analysis
• A novel learning-based frame pooling method for Event Detection
• Modelling Temporal Information Using Discrete Fourier Transform for Recognizing Emotions in User-generated Videos
• Modelling Temporal Information Using Discrete Fourier Transform for Video Classification
• cvpaper.challenge in 2016: Futuristic Computer Vision through 1,600 Papers Survey
• Bidirectional Multirate Reconstruction for Temporal Modeling in Videos
• cvpaper.challenge in 2015 - A review of CVPR2015 and DeepSurvey