Video Action Understanding
arXiv:2010.06647 · doi:10.1109/ACCESS.2021.3115476
Abstract
Many believe that the successes of deep learning on image understanding problems can be replicated in the realm of video understanding. However, due to the scale and temporal nature of video, the span of video understanding problems and the set of proposed deep learning solutions is arguably wider and more diverse than those of their 2D image siblings. Finding, identifying, and predicting actions are a few of the most salient tasks in this emerging and rapidly evolving field. With a pedagogical emphasis, this tutorial introduces and systematizes fundamental topics, basic concepts, and notable examples in supervised video action understanding. Specifically, we clarify a taxonomy of action problems, catalog and highlight video datasets, describe common video data preparation methods, present the building blocks of state-of-the art deep learning model architectures, and formalize domain-specific metrics to baseline proposed solutions. This tutorial is intended to be accessible to a general computer science audience and assumes a conceptual understanding of supervised learning.
arXiv commentAccepted for publication in IEEE Access
References in corpus (49)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
- Two-Stream Convolutional Networks for Action Recognition in Videos
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Instance Normalization: The Missing Ingredient for Fast Stylization
- The Kinetics Human Action Video Dataset
- Deep Learning for Sensor-based Activity Recognition: A Survey
- NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
- Is Space-Time Attention All You Need for Video Understanding?
- Action Recognition with Trajectory-Pooled Deep-Convolutional Descriptors
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- Single Shot Temporal Action Detection
- A Short Note on the Kinetics-700-2020 Human Action Dataset
- A Short Note about Kinetics-600
- Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research
- CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016
- Advances in Human Action Recognition: A Survey
- You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localization
- Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos
- A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation
- The AVA-Kinetics Localized Human Actions Video Dataset
- Temporal Convolution Based Action Proposal: Submission to ActivityNet 2017
- Multiscale Vision Transformers
- Revisiting the Effectiveness of Off-the-shelf Temporal Modeling Approaches for Large-scale Video Classification
- The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary
- Review of Action Recognition and Detection Methods
- ActivityNet Challenge 2017 Summary
- On the effectiveness of task granularity for transfer learning
- Video action detection by learning graph-based spatio-temporal interactions
- Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection
- Deep Learning for Vision-based Prediction: A Survey
- AViD Dataset: Anonymized Videos from Diverse Countries
- Exploiting Spatial-Temporal Modelling and Multi-Modal Fusion for Human Action Recognition
- Baidu-UTS Submission to the EPIC-Kitchens Action Recognition Challenge 2019
- Proposal Relation Network for Temporal Action Detection
- Spatio-temporal Action Recognition: A Survey
- Spatio-temporal Human Action Localisation and Instance Segmentation in Temporally Untrimmed Videos
- Trimmed Action Recognition, Dense-Captioning Events in Videos, and Spatio-temporal Action Localization with Focus on ActivityNet Challenge 2019
- Relation Modeling in Spatio-Temporal Action Localization
- Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding
- Learning Sparse 2D Temporal Adjacent Networks for Temporal Action Localization
- CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)
- TinyAction Challenge: Recognizing Real-world Low-resolution Activities in Videos
- 1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
- Exploring Stronger Feature for Temporal Action Localization
- Temporal Fusion Network for Temporal Action Localization:Submission to ActivityNet Challenge 2020 (Task E)
- Top-1 Solution of Multi-Moments in Time Challenge 2019
Cited by in corpus (4)
- Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition
- ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition
- Performance Evaluation of Action Recognition Models on Low Quality Videos
- Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition