Relational Action Forecasting
arXiv:1904.04231
Abstract
This paper focuses on multi-person action forecasting in videos. More precisely, given a history of H previous frames, the goal is to detect actors and to predict their future actions for the next T frames. Our approach jointly models temporal and spatial interactions among different actors by constructing a recurrent graph, using actor proposals obtained with Faster R-CNN as nodes. Our method learns to select a subset of discriminative relations without requiring explicit supervision, thus enabling us to tackle challenging visual data. We refer to our model as Discriminative Relational Recurrent Network (DRRN). Evaluation of action prediction on AVA demonstrates the effectiveness of our proposed method compared to simpler baselines. Furthermore, we significantly improve performance on the task of early action classification on J-HMDB, from the previous SOTA of 48% to 60%.
CVPR 2019 (oral)
References in corpus (30)
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- The Kinetics Human Action Video Dataset
- SGDR: Stochastic Gradient Descent with Warm Restarts
- YouTube-8M: A Large-Scale Video Classification Benchmark
- Generating Videos with Scene Dynamics
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- A simple neural network module for relational reasoning
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Video (language) modeling: a baseline for generative models of natural videos
- Unsupervised Learning for Physical Interaction through Video Prediction
- Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
- Learning to Generate Long-term Future via Hierarchical Prediction
- Speed/accuracy trade-offs for modern convolutional object detectors
- Non-local Neural Networks
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
- Object Level Visual Reasoning in Videos
- DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
- Learning to track for spatio-temporal action localization
- Recurrent Network Models for Human Dynamics
- Stochastic Variational Video Prediction
- Relation Networks for Object Detection
- Detecting Visual Relationships with Deep Relational Networks
- Predicting Deeper into the Future of Semantic Segmentation
- Learning to Detect Human-Object Interactions
- Relational recurrent neural networks
- Recurrent Relational Networks
- Actor-Centric Relation Network
- Motion Prediction Under Multimodality with Conditional Stochastic Networks
- CAR-Net: Clairvoyant Attentive Recurrent Network