Fully-Coupled Two-Stream Spatiotemporal Networks for Extremely Low Resolution Action Recognition

Abstract

A major emerging challenge is how to protect people's privacy as cameras and computer vision are increasingly integrated into our daily lives, including in smart devices inside homes. A potential solution is to capture and record just the minimum amount of information needed to perform a task of interest. In this paper, we propose a fully-coupled two-stream spatiotemporal architecture for reliable human action recognition on extremely low resolution (e.g., 12x16 pixel) videos. We provide an efficient method to extract spatial and temporal features and to aggregate them into a robust feature representation for an entire action video sequence. We also consider how to incorporate high resolution videos during training in order to build better low resolution action recognition models. We evaluate on two publicly-available datasets, showing significant improvements over the state-of-the-art.

9 pagers, 5 figures, published in WACV 2018

References in corpus (7)

• Convolutional Two-Stream Network Fusion for Video Action Recognition
• Learning Spatiotemporal Features with 3D Convolutional Networks
• Hidden Two-Stream Convolutional Networks for Action Recognition
• TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals
• Privacy-Preserving Human Activity Recognition from Extreme Low Resolution
• Studying Very Low Resolution Recognition Using Deep Networks
• Deep Temporal Linear Encoding Networks

Cited by in corpus (2)

• Privacy-Preserving Action Recognition for Smart Hospitals using Low-Resolution Depth Images
• Cross-Enhancement Transform Two-Stream 3D ConvNets for Action Recognition