T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
arXiv:1604.02532 · doi:10.1109/TCSVT.2017.2736553
Abstract
The state-of-the-art performance for object detection has been significantly improved over the past two years. Besides the introduction of powerful deep neural networks such as GoogleNet and VGG, novel object detection frameworks such as R-CNN and its successors, Fast R-CNN and Faster R-CNN, play an essential role in improving the state-of-the-art. Despite their effectiveness on still images, those frameworks are not specifically designed for object detection from videos. Temporal and contextual information of videos are not fully investigated and utilized. In this work, we propose a deep learning framework that incorporates temporal and contextual information from tubelets obtained in videos, which dramatically improves the baseline performance of existing still-image detection frameworks when they are applied to videos. It is called T-CNN, i.e. tubelets with convolutional neueral networks. The proposed framework won the recently introduced object-detection-from-video (VID) task with provided data in the ImageNet Large-Scale Visual Recognition Challenge 2015 (ILSVRC2015).
arXiv commentImageNet 2015 VID challenge tech report. The first two authors share co-first authorship. Accepted as a Transaction paper by T-CSVT Special Issue on Large Scale and Nonlinear Similarity Learning for Intelligent Video Analysis
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- SSD: Single Shot MultiBox Detector
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Fully Convolutional Networks for Semantic Segmentation
- Cost-Effective Active Learning for Deep Image Classification
- Object Detection from Video Tubelets with Convolutional Neural Networks
- Object Detection in Videos with Tubelet Proposal Networks
- Seq-NMS for Video Object Detection
- DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
- Learning Multi-Domain Convolutional Neural Networks for Visual Tracking
- Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
- Window-Object Relationship Guided Representation Learning for Generic Object Detections
Cited by in corpus (72)
- GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
- A Survey of Deep Learning-based Object Detection
- Tracking without bells and whistles
- Object Detection in 20 Years: A Survey
- Deep Learning for UAV-based Object Detection and Tracking: A Survey
- TransTrack: Multiple Object Tracking with Transformer
- A Comprehensive Survey of Scene Graphs: Generation and Application
- TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers
- Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks
- Flow-Guided Feature Aggregation for Video Object Detection
- Multi-Class Multi-Object Tracking using Changing Point Detection
- Real-World Image Datasets for Federated Learning
- Person Search with Natural Language Description
- MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection
- Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
- NoScope: Optimizing Neural Network Queries over Video at Scale
- Cascaded Revision Network for Novel Object Captioning
- Mobile Video Object Detection with Temporally-Aware Feature Maps
- QueryProp: Object Query Propagation for High-Performance Video Object Detection
- Sequence Level Semantics Aggregation for Video Object Detection
- AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling
- Towards High Performance Video Object Detection for Mobiles
- Looking Fast and Slow: Memory-Guided Mobile Video Object Detection
- Relation Distillation Networks for Video Object Detection
- Impression Network for Video Object Detection
- Dual Semantic Fusion Network for Video Object Detection
- Differentiating Objects by Motion: Joint Detection and Tracking of Small Flying Objects
- TDIOT: Target-driven Inference for Deep Video Object Tracking
- Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation
- A semi-supervised self-training method to develop assistive intelligence for segmenting multiclass bridge elements from inspection videos
- Progressive Sparse Local Attention for Video object detection
- Exploit the Connectivity: Multi-Object Tracking with TrackletNet
- TrackNet: Simultaneous Object Detection and Tracking and Its Application in Traffic Video Analysis
- A Delay Metric for Video Object Detection: What Average Precision Fails to Tell
- On The Stability of Video Detection and Tracking
- Tracking Objects as Points
- Efficient One-stage Video Object Detection by Exploiting Temporal Consistency
- ACDnet: An action detection network for real-time edge computing based on flow-guided feature approximation and memory aggregation
- Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector
- Detection and Tracking Meet Drones Challenge
- Pack and Detect: Fast Object Detection in Videos Using Region-of-Interest Packing
- TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model
- ApproxNet: Content and Contention-Aware Video Analytics System for Embedded Clients
- A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
- Object Detection in Video with Spatial-temporal Context Aggregation
- Recurrent Neural Networks for video object detection
- Fast Object Detection in Compressed Video
- ApproxDet: Content and Contention-Aware Approximate Object Detection for Mobiles
- Object Detection in Videos by High Quality Object Linking
- End-to-End Video Object Detection with Spatial-Temporal Transformers
- Towards High Performance Video Object Detection
- Spatiotemporal Object Detection for Improved Aerial Vehicle Detection in Traffic Monitoring
- Attention Mechanisms for Object Recognition with Event-Based Cameras
- Learning Motion in Feature Space: Locally-Consistent Deformable Convolution Networks for Fine-Grained Action Detection
- Object-aware Feature Aggregation for Video Object Detection
- ViP-CNN: Visual Phrase Guided Convolutional Neural Network
- TYolov5: A Temporal Yolov5 Detector Based on Quasi-Recurrent Neural Networks for Real-Time Handgun Detection in Video
- Learning to Track Object Position through Occlusion
- Finding a Needle in a Haystack: Tiny Flying Object Detection in 4K Videos using a Joint Detection-and-Tracking Approach
- Look at here : Utilizing supervision to attend subtle key regions
- Great Ape Detection in Challenging Jungle Camera Trap Footage via Attention-Based Spatial and Temporal Feature Blending
- SCNN: A General Distribution based Statistical Convolutional Neural Network with Application to Video Object Detection
- Adversarial Learning of Label Dependency: A Novel Framework for Multi-class Classification
- Mobiprox: Supporting Dynamic Approximate Computing on Mobiles
- Minimum Delay Object Detection From Video
- Class agnostic moving target detection by color and location prediction of moving area
- Spatio-Temporal Context Modeling for Road Obstacle Detection
- A Generic Object Re-identification System for Short Videos
- 3D Object Detection and Tracking Based on Streaming Data
- Spatio-temporal Tubelet Feature Aggregation and Object Linking in Videos
- Long Short-Term Relation Networks for Video Action Detection
- Single Shot Video Object Detector