Improving Weakly-supervised Object Localization via Causal Intervention
arXiv:2104.10351
Abstract
The recent emerged weakly supervised object localization (WSOL) methods can learn to localize an object in the image only using image-level labels. Previous works endeavor to perceive the interval objects from the small and sparse discriminative attention map, yet ignoring the co-occurrence confounder (e.g., bird and sky), which makes the model inspection (e.g., CAM) hard to distinguish between the object and context. In this paper, we make an early attempt to tackle this challenge via causal intervention (CI). Our proposed method, dubbed CI-CAM, explores the causalities among images, contexts, and categories to eliminate the biased co-occurrence in the class activation maps thus improving the accuracy of object localization. Extensive experiments on several benchmarks demonstrate the effectiveness of CI-CAM in learning the clear object boundaries from confounding contexts. Particularly, in CUB-200-2011 which severely suffers from the co-occurrence confounder, CI-CAM significantly outperforms the traditional CAM-based baseline (58.39% vs 52.4% in top-1 localization accuracy). While in more general scenarios such as ImageNet, CI-CAM can also perform on par with the state of the arts.
arXiv comment9 pages, 5 figures. This paper was accepted by ACM Multimedia 2021. The code can be available at https://github.com/shaofeifei11/CI-CAM
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Interventional Few-Shot Learning
- Learning Independent Causal Mechanisms
- Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
- Two-Phase Learning for Weakly Supervised Object Localization
- Counterfactuals uncover the modular structure of deep generative models
- A Survey on Causal Inference
- Counterfactual VQA: A Cause-Effect Look at Language Bias
- Deep Learning for Weakly-Supervised Object Detection and Object Localization: A Survey