Deep Learning for Video Classification and Captioning
arXiv:1609.06782 · doi:10.1145/3122865.3122867
Abstract
Accelerated by the tremendous increase in Internet bandwidth and storage space, video data has been generated, published and spread explosively, becoming an indispensable part of today's big data. In this paper, we focus on reviewing two lines of research aiming to stimulate the comprehension of videos with deep learning: video classification and video captioning. While video classification concentrates on automatically labeling video clips based on their semantic contents like human actions or complex events, video captioning attempts to generate a complete and natural sentence, enriching the single label as in video classification, to capture the most informative dynamics in videos. In addition, we also provide a review of popular benchmarks and competitions, which are critical for evaluating the technical progress of this vibrant field.
Book chapter in Frontiers of Multimedia Research
References in corpus (10)
- Understanding individual human mobility patterns
- Two-Stream Convolutional Networks for Action Recognition in Videos
- YFCC100M: The New Data in Multimedia Research
- The THUMOS Challenge on Action Recognition for Videos "in the Wild"
- F-formation Detection: Individuating Free-standing Conversational Groups in Images
- Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data
- EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis
- Exploiting Image-trained CNN Architectures for Unconstrained Video Classification
- An On-line Variational Bayesian Model for Multi-Person Tracking from Cluttered Scenes
- Vision-Guided Robot Hearing