FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
arXiv:2004.06704
Abstract
On public benchmarks, current action recognition techniques have achieved great success. However, when used in real-world applications, e.g. sport analysis, which requires the capability of parsing an activity into phases and differentiating between subtly different actions, their performances remain far from being satisfactory. To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnastic videos. Compared to existing action recognition datasets, FineGym is distinguished in richness, quality, and diversity. In particular, it provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy. For example, a "balance beam" event will be annotated as a sequence of elementary sub-actions derived from five sets: "leap-jump-hop", "beam-turns", "flight-salto", "flight-handspring", and "dismount", where the sub-action in each set will be further annotated with finely defined class labels. This new level of granularity presents significant challenges for action recognition, e.g. how to parse the temporal structures from a coherent action, and how to distinguish between subtly different action classes. We systematically investigate representative methods on this dataset and obtain a number of interesting findings. We hope this dataset could advance research towards action understanding.
CVPR 2020 Oral (3 strong accepts); Project page: https://sdolivia.github.io/FineGym/
References in corpus (5)
- Two-Stream Convolutional Networks for Action Recognition in Videos
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification
- CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark
Cited by in corpus (5)
- TSA-Net: Tube Self-Attention Network for Action Quality Assessment
- Three-Stream 3D/1D CNN for Fine-Grained Action Classification and Segmentation in Table Tennis
- Joint Learning On The Hierarchy Representation for Fine-Grained Human Action Recognition
- Intra- and Inter-Action Understanding via Temporal Action Parsing
- Efficient Modelling Across Time of Human Actions and Interactions