Temporal Attentive Alignment for Large-Scale Video Domain Adaptation

Abstract

Although various image-based domain adaptation (DA) techniques have been proposed in recent years, domain shift in videos is still not well-explored. Most previous works only evaluate performance on small-scale datasets which are saturated. Therefore, we first propose two large-scale video DA datasets with much larger domain discrepancy: UCF-HMDB_full and Kinetics-Gameplay. Second, we investigate different DA integration methods for videos, and show that simultaneously aligning and learning temporal dynamics achieves effective alignment even without sophisticated DA methods. Finally, we propose Temporal Attentive Adversarial Adaptation Network (TA3N), which explicitly attends to the temporal dynamics using domain discrepancy for more effective domain alignment, achieving state-of-the-art performance on four video DA datasets (e.g. 7.9% accuracy gain over "Source only" from 73.9% to 81.8% on "HMDB --> UCF", and 10.3% gain on "Kinetics --> Gameplay"). The code and data are released at http://github.com/cmhungsteve/TA3N.

ICCV 2019 (Oral) camera-ready + supplementary. Code and data: http://github.com/cmhungsteve/TA3N

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Two-Stream Convolutional Networks for Action Recognition in Videos
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• The Kinetics Human Action Video Dataset
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Revisiting Batch Normalization For Practical Domain Adaptation
• A Short Note about Kinetics-600
• Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
• Learning to cluster in order to transfer across domains and tasks
• Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation

Cited by in corpus (5)

• Adversarial Cross-Domain Action Recognition with Co-Attention
• Unsupervised Domain Adaptation for Video Semantic Segmentation
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
• Generative Adversarial Networks for Video-to-Video Domain Adaptation
• Channel-Temporal Attention for First-Person Video Domain Adaptation