Fast Video Shot Transition Localization with Deep Structured Models
arXiv:1808.04234
Abstract
Detection of video shot transition is a crucial pre-processing step in video analysis. Previous studies are restricted on detecting sudden content changes between frames through similarity measurement and multi-scale operations are widely utilized to deal with transitions of various lengths. However, localization of gradual transitions are still under-explored due to the high visual similarity between adjacent frames. Cut shot transitions are abrupt semantic breaks while gradual shot transitions contain low-level spatial-temporal patterns caused by video effects in addition to the gradual semantic breaks, e.g. dissolve. In order to address the problem, we propose a structured network which is able to detect these two shot transitions using targeted models separately. Considering speed performance trade-offs, we design a smart framework. With one TITAN GPU, the proposed method can achieve a 30\(\times\) real-time speed. Experiments on public TRECVID07 and RAI databases show that our method outperforms the state-of-the-art methods. In order to train a high-performance shot transition detector, we contribute a new database ClipShots, which contains 128636 cut transitions and 38120 gradual transitions from 4039 online videos. ClipShots intentionally collect short videos for more hard cases caused by hand-held camera vibrations, large object motions, and occlusion.
arXiv comment16 pages, 3 figures, submitted to ACCV
References in corpus (10)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- The Kinetics Human Action Video Dataset
- Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
- Learning to Compare Image Patches via Convolutional Neural Networks
- Temporal Action Detection with Structured Segment Networks
- R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
- Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
- Learning Fine-grained Image Similarity with Deep Ranking
- Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks