Evidential Deep Learning for Open Set Action Recognition

Abstract

In a real-world scenario, human actions are typically out of the distribution from training data, which requires a model to both recognize the known actions and reject the unknown. Different from image data, video actions are more challenging to be recognized in an open-set setting due to the uncertain temporal dynamics and static bias of human actions. In this paper, we propose a Deep Evidential Action Recognition (DEAR) method to recognize actions in an open testing set. Specifically, we formulate the action recognition problem from the evidential deep learning (EDL) perspective and propose a novel model calibration method to regularize the EDL training. Besides, to mitigate the static bias of video representation, we propose a plug-and-play module to debias the learned representation through contrastive learning. Experimental results show that our DEAR method achieves consistent performance gain on multiple mainstream action recognition models and benchmarks. Code and pre-trained models are available at {\small{\url{https://www.rit.edu/actionlab/dear}}}.

ICCV 2021 Oral

References in corpus (10)

• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Recent Advances in Open Set Recognition: A Survey
• Evidential Deep Learning to Quantify Classification Uncertainty
• Bayesian Active Learning for Classification and Preference Learning
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
• Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition
• Learning De-biased Representations with Biased Representations
• Improving model calibration with accuracy versus uncertainty optimization
• BAR: Bayesian Activity Recognition using variational inference