Dynamic Graph Modules for Modeling Object-Object Interactions in Activity Recognition
arXiv:1812.05637
Abstract
Video action recognition, a critical problem in video understanding, has been gaining increasing attention. To identify actions induced by complex object-object interactions, we need to consider not only spatial relations among objects in a single frame, but also temporal relations among different or the same objects across multiple frames. However, existing approaches that model video representations and non-local features are either incapable of explicitly modeling relations at the object-object level or unable to handle streaming videos. In this paper, we propose a novel dynamic hidden graph module to model complex object-object interactions in videos, of which two instantiations are considered: a visual graph that captures appearance/motion changes among objects and a location graph that captures relative spatiotemporal position changes among objects. Additionally, the proposed graph module allows us to process streaming videos, setting it apart from existing methods. Experimental results on benchmark datasets, Something-Something and ActivityNet, show the competitive performance of our method.
arXiv comment14 pages, 5 figures
References in corpus (15)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Improving neural networks by preventing co-adaptation of feature detectors
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Geometric deep learning: going beyond Euclidean data
- The Kinetics Human Action Video Dataset
- Few-Shot Learning with Graph Neural Networks
- ConvNet Architecture Search for Spatiotemporal Feature Learning
- TSM: Temporal Shift Module for Efficient Video Understanding
- Non-local Neural Networks
- Visual Interaction Networks
- Learning Human-Object Interactions by Graph Parsing Neural Networks
- On the effectiveness of task granularity for transfer learning
- Appearance-and-Relation Networks for Video Classification
- Graph-Based Global Reasoning Networks
- Attend and Interact: Higher-Order Object Interactions for Video Understanding