Treatment Learning Causal Transformer for Noisy Image Classification
arXiv:2203.15529 · doi:10.1109/WACV56688.2023.00608
Abstract
Current top-notch deep learning (DL) based vision models are primarily based on exploring and exploiting the inherent correlations between training data samples and their associated labels. However, a known practical challenge is their degraded performance against "noisy" data, induced by different circumstances such as spurious correlations, irrelevant contexts, domain shift, and adversarial attacks. In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy by jointly estimating their treatment effects. Motivated from causal variational inference, we propose a transformer-based architecture, Treatment Learning Causal Transformer (TLT), that uses a latent generative model to estimate robust feature representations from current observational input for noise image classification. Depending on the estimated noise level (modeled as a binary treatment factor), TLT assigns the corresponding inference network trained by the designed causal loss for prediction. We also create new noisy image datasets incorporating a wide range of noise factors (e.g., object masking, style transfer, and adversarial perturbation) for performance benchmarking. The superior performance of TLT in noisy image classification is further validated by several refutation evaluation metrics. As a by-product, TLT also improves visual salience methods for perceiving noisy images.
arXiv commentAccepted to IEEE WACV 2023. The first version was finished in May 2018
References in corpus (14)
- Auto-Encoding Variational Bayes
- Explaining and Harnessing Adversarial Examples
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
- Instance Normalization: The Missing Ingredient for Fast Stylization
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution
- A large annotated medical image dataset for the development and evaluation of segmentation algorithms
- Deep Bayesian Active Learning with Image Data
- Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (1995)
- MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs
- CAM: Causal additive models, high-dimensional order search and penalized regression
- Causal Discovery with Continuous Additive Noise Models
- Visual Causal Feature Learning
- When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networks