Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks

Abstract

This thesis explore different approaches using Convolutional and Recurrent Neural Networks to classify and temporally localize activities on videos, furthermore an implementation to achieve it has been proposed. As the first step, features have been extracted from video frames using an state of the art 3D Convolutional Neural Network. This features are fed in a recurrent neural network that solves the activity classification and temporally location tasks in a simple and flexible way. Different architectures and configurations have been tested in order to achieve the best performance and learning of the video dataset provided. In addition it has been studied different kind of post processing over the trained network's output to achieve a better results on the temporally localization of activities on the videos. The results provided by the neural network developed in this thesis have been submitted to the ActivityNet Challenge 2016 of the CVPR, achieving competitive results using a simple and flexible architecture.

arXiv commentBest Poster Award at the 1st NIPS Workshop on Large Scale Computer Vision Systems (Barcelona, December 2016). Source code available at https://imatge-upc.github.io/activitynet-2016-cvprw/

References in corpus (9)

• Towards Good Practices for Very Deep Two-Stream ConvNets
• Describing Videos by Exploiting Temporal Structure
• Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
• Learning Spatiotemporal Features with 3D Convolutional Networks
• Delving Deeper into Convolutional Networks for Learning Video Representations
• Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos
• Contextual Action Recognition with R*CNN
• End-to-end Learning of Action Detection from Frame Glimpses in Videos
• Deep End2End Voxel2Voxel Prediction

Cited by in corpus (17)

• Temporal Action Detection with Structured Segment Networks
• A Pursuit of Temporal Accuracy in General Activity Detection
• R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
• Temporal Context Network for Activity Localization in Videos
• Weakly Supervised Action Localization by Sparse Temporal Pooling Network
• Graph Convolutional Networks for Temporal Action Localization
• Local-Global Video-Text Interactions for Temporal Grounding
• Simple vs complex temporal recurrences for video saliency prediction
• SA-Net: Deep Neural Network for Robot Trajectory Recognition from RGB-D Streams
• Similarity R-C3D for Few-shot Temporal Activity Detection
• Revisiting Few-shot Activity Detection with Class Similarity Control
• Budget-Aware Activity Detection with A Recurrent Policy Network
• Graph Distillation for Action Detection with Privileged Modalities
• Two-Stream Region Convolutional 3D Network for Temporal Activity Detection
• Video Summarization via Actionness Ranking
• Graph Convolutional Module for Temporal Action Localization in Videos
• Online Spatiotemporal Action Detection and Prediction via Causal Representations