Object Detection in Video with Spatiotemporal Sampling Networks

Abstract

We propose a Spatiotemporal Sampling Network (STSN) that uses deformable convolutions across time for object detection in videos. Our STSN performs object detection in a video frame by learning to spatially sample features from the adjacent frames. This naturally renders the approach robust to occlusion or motion blur in individual frames. Our framework does not require additional supervision, as it optimizes sampling locations directly with respect to object detection performance. Our STSN outperforms the state-of-the-art on the ImageNet VID dataset and compared to prior video object detection methods it uses a simpler design, and does not require optical flow data for training.

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Going Deeper with Convolutions
• YOLO9000: Better, Faster, Stronger
• Unsupervised Learning for Physical Interaction through Video Prediction

Cited by in corpus (10)

• TDAN: Temporally Deformable Alignment Network for Video Super-Resolution
• EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
• MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection
• QueryProp: Object Query Propagation for High-Performance Video Object Detection
• Learning Discriminative Motion Features Through Detection
• Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation
• Fast Object Detection in Compressed Video
• Plug & Play Convolutional Regression Tracker for Video Object Detection
• Spatio-temporal Tubelet Feature Aggregation and Object Linking in Videos
• Efficient Video Understanding via Layered Multi Frame-Rate Analysis