Marginalized Average Attentional Network for Weakly-Supervised Learning

Abstract

In weakly-supervised temporal action localization, previous works have failed to locate dense and integral regions for each entire action due to the overestimation of the most salient regions. To alleviate this issue, we propose a marginalized average attentional network (MAAN) to suppress the dominant response of the most salient regions in a principled manner. The MAAN employs a novel marginalized average aggregation (MAA) module and learns a set of latent discriminative probabilities in an end-to-end fashion. MAA samples multiple subsets from the video snippet features according to a set of latent discriminative probabilities and takes the expectation over all the averaged subset features. Theoretically, we prove that the MAA module with learned latent discriminative probabilities successfully reduces the difference in responses between the most salient regions and the others. Therefore, MAAN is able to generate better class activation sequences and identify dense and integral action regions in the videos. Moreover, we propose a fast algorithm to reduce the complexity of constructing MAA from O($2^T$) to O($T^2$). Extensive experiments on two large-scale video datasets show that our MAAN achieves superior performance on weakly-supervised temporal action localization

Cited by in corpus (13)

• Curriculum Loss: Robust Learning and Generalization against Label Corruption
• ACM-Net: Action Context Modeling Network for Weakly-Supervised Temporal Action Localization
• Slow Motion Matters: A Slow Motion Enhanced Network for Weakly Supervised Temporal Action Localization
• Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning
• Weakly-Supervised Action Localization by Generative Attention Modeling
• Improving Weakly Supervised Temporal Action Localization by Exploiting Multi-resolution Information in Temporal Domain
• Action Shuffling for Weakly Supervised Temporal Localization
• ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization
• Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization
• WOAD: Weakly Supervised Online Action Detection in Untrimmed Videos
• Not All Attention Is Needed: Gated Attention Network for Sequence Data
• Weakly Supervised Action Selection Learning in Video
• CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning