TSA-Net: Tube Self-Attention Network for Action Quality Assessment

Abstract

In recent years, assessing action quality from videos has attracted growing attention in computer vision community and human computer interaction. Most existing approaches usually tackle this problem by directly migrating the model from action recognition tasks, which ignores the intrinsic differences within the feature map such as foreground and background information. To address this issue, we propose a Tube Self-Attention Network (TSA-Net) for action quality assessment (AQA). Specifically, we introduce a single object tracker into AQA and propose the Tube Self-Attention Module (TSA), which can efficiently generate rich spatio-temporal contextual information by adopting sparse feature interactions. The TSA module is embedded in existing video networks to form TSA-Net. Overall, our TSA-Net is with the following merits: 1) High computational efficiency, 2) High flexibility, and 3) The state-of-the art performance. Extensive experiments are conducted on popular action quality assessment datasets including AQA-7 and MTL-AQA. Besides, a dataset named Fall Recognition in Figure Skating (FR-FS) is proposed to explore the basic action assessment in the figure skating scene.

9 pages, 7 figures, conference paper

References in corpus (5)

• The Kinetics Human Action Video Dataset
• Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
• FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
• Feature Pyramid Transformer
• What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

Cited by in corpus (1)

• A Survey of Video-based Action Quality Assessment