CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016

Abstract

This paper presents the method that underlies our submission to the untrimmed video classification task of ActivityNet Challenge 2016. We follow the basic pipeline of temporal segment networks and further raise the performance via a number of other techniques. Specifically, we use the latest deep model architecture, e.g., ResNet and Inception V3, and introduce new aggregation schemes (top-k and attention-weighted pooling). Additionally, we incorporate the audio as a complementary channel, extracting relevant information via a CNN applied to the spectrograms. With these techniques, we derive an ensemble of deep models, which, together, attains a high classification accuracy (mAP $93.23\%$) on the testing set and secured the first place in the challenge.

References in corpus (4)

• Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice
• Deep Convolutional Neural Networks and Data Augmentation for Acoustic Event Detection
• Real-time Action Recognition with Enhanced Motion Vector CNNs
• Learning Multiscale Features Directly From Waveforms

Cited by in corpus (43)

• End-to-end Temporal Action Detection with Transformer
• Revisiting Anchor Mechanisms for Temporal Action Localization
• BSN: Boundary Sensitive Network for Temporal Action Proposal Generation
• BMN: Boundary-Matching Network for Temporal Action Proposal Generation
• Temporal Convolution Based Action Proposal: Submission to ActivityNet 2017
• Cascaded Boundary Regression for Temporal Action Detection
• End-to-End Dense Video Captioning with Masked Transformer
• PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
• Temporally smooth online action detection using cycle-consistent future anticipation
• Video Action Understanding
• SRG: Snippet Relatedness-based Temporal Action Proposal Generator
• G-TAD: Sub-Graph Localization for Temporal Action Detection
• Video-based Human Action Recognition using Deep Learning: A Review
• RED: Reinforced Encoder-Decoder Networks for Action Anticipation
• Motion-Appearance Co-Memory Networks for Video Question Answering
• BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation
• Temporal Recurrent Networks for Online Action Detection
• CTAP: Complementary Temporal Action Proposal Generation
• Fast Learning of Temporal Action Proposal via Dense Boundary Generator
• ABN: Agent-Aware Boundary Networks for Temporal Action Proposal Generation
• Temporal Context Aggregation Network for Temporal Action Proposal Refinement
• AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation
• MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
• Boundary Content Graph Neural Network for Temporal Action Proposal Generation
• Video Self-Stitching Graph Network for Temporal Action Localization
• Bottom-Up Temporal Action Localization with Mutual Regularization
• Deep Concept-wise Temporal Convolutional Networks for Action Localization
• Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
• Multi-shot Temporal Event Localization: a Benchmark
• Dense-Captioning Events in Videos: SYSU Submission to ActivityNet Challenge 2020
• Winning the CVPR'2021 Kinetics-GEBD Challenge: Contrastive Learning Approach
• Scale Matters: Temporal Scale Aggregation Network for Precise Action Localization in Untrimmed Videos
• Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization
• Grounded Video Description
• Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021
• Multi-Level Temporal Pyramid Network for Action Detection
• Understanding Action Sequences based on Video Captioning for Learning-from-Observation
• CMSN: Continuous Multi-stage Network and Variable Margin Cosine Loss for Temporal Action Proposal Generation
• Activity Recognition on a Large Scale in Short Videos - Moments in Time Dataset
• Exploring Feature Representation and Training strategies in Temporal Action Localization
• Graph Convolutional Module for Temporal Action Localization in Videos
• Complementary Boundary Generator with Scale-Invariant Relation Modeling for Temporal Action Localization: Submission to ActivityNet Challenge 2020
• Information Elevation Network for Fast Online Action Detection