Diagnosing Error in Temporal Action Detectors
arXiv:1807.10706
Abstract
Despite the recent progress in video understanding and the continuous rate of improvement in temporal action localization throughout the years, it is still unclear how far (or close?) we are to solving the problem. To this end, we introduce a new diagnostic tool to analyze the performance of temporal action detectors in videos and compare different methods beyond a single scalar metric. We exemplify the use of our tool by analyzing the performance of the top rewarded entries in the latest ActivityNet action localization challenge. Our analysis shows that the most impactful areas to work on are: strategies to better handle temporal context around the instances, improving the robustness w.r.t. the instance absolute and relative size, and strategies to reduce the localization errors. Moreover, our experimental analysis finds the lack of agreement among annotator is not a major roadblock to attain progress in the field. Our diagnostic tool is publicly available to keep fueling the minds of other researchers with additional insights about their algorithms.
arXiv commentAccepted to ECCV 2018
References in corpus (13)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Deep Residual Learning for Image Recognition
- The Kinetics Human Action Video Dataset
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- Non-local Neural Networks
- Temporal Action Detection with Structured Segment Networks
- A Pursuit of Temporal Accuracy in General Activity Detection
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
- Temporal Convolution Based Action Proposal: Submission to ActivityNet 2017
- ActivityNet Challenge 2017 Summary
- Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video