Long-Term Feature Banks for Detailed Video Understanding
arXiv:1812.05038
Abstract
To understand the world, we humans constantly need to relate the present to the past, and put events in context. In this paper, we enable existing video models to do the same. We propose a long-term feature bank---supportive information extracted over the entire span of a video---to augment state-of-the-art video models that otherwise would only view short clips of 2-5 seconds. Our experiments demonstrate that augmenting 3D convolutional networks with a long-term feature bank yields state-of-the-art results on three challenging video datasets: AVA, EPIC-Kitchens, and Charades.
arXiv commentCode and models are available at https://github.com/facebookresearch/video-long-term-feature-banks
References in corpus (12)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- Spatiotemporal Residual Networks for Video Action Recognition
- ConvNet Architecture Search for Spatiotemporal Feature Learning
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- Learnable pooling with Context Gating for video classification
- Object Level Visual Reasoning in Videos
- Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding
- A Better Baseline for AVA
- Non-local NetVLAD Encoding for Video Classification