Channel-Temporal Attention for First-Person Video Domain Adaptation

Abstract

Unsupervised Domain Adaptation (UDA) can transfer knowledge from labeled source data to unlabeled target data of the same categories. However, UDA for first-person action recognition is an under-explored problem, with lack of datasets and limited consideration of first-person video characteristics. This paper focuses on addressing this problem. Firstly, we propose two small-scale first-person video domain adaptation datasets: ADL${}_{small}$ and GTEA-KITCHEN. Secondly, we introduce channel-temporal attention blocks to capture the channel-wise and temporal-wise relationships and model their inter-dependencies important to first-person vision. Finally, we propose a Channel-Temporal Attention Network (CTAN) to integrate these blocks into existing architectures. CTAN outperforms baselines on the two proposed datasets and one existing dataset EPIC${}_{cvpr20}$.

References in corpus (8)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• The Kinetics Human Action Video Dataset
• Conditional Adversarial Domain Adaptation
• Convolutional Two-Stream Network Fusion for Video Action Recognition
• ConvNet Architecture Search for Spatiotemporal Feature Learning
• Temporal Relational Reasoning in Videos
• Self-Training and Adversarial Background Regularization for Unsupervised Domain Adaptive One-Stage Object Detection
• Temporal Attentive Alignment for Large-Scale Video Domain Adaptation