Towards Robust Classification Model by Counterfactual and Invariant Data Generation

Abstract

Despite the success of machine learning applications in science, industry, and society in general, many approaches are known to be non-robust, often relying on spurious correlations to make predictions. Spuriousness occurs when some features correlate with labels but are not causal; relying on such features prevents models from generalizing to unseen environments where such correlations break. In this work, we focus on image classification and propose two data generation processes to reduce spuriousness. Given human annotations of the subset of the features responsible (causal) for the labels (e.g. bounding boxes), we modify this causal set to generate a surrogate image that no longer has the same label (i.e. a counterfactual image). We also alter non-causal features to generate images still recognized as the original labels, which helps to learn a model invariant to these features. In several challenging datasets, our data generations outperform state-of-the-art methods in accuracy when spurious correlations break, and increase the saliency focus on causal features providing better explanations.

arXiv commentAccepted in 2021 CVPR

References in corpus (9)

• Shortcut Learning in Deep Neural Networks
• Confounding variables can degrade generalization performance of radiological deep learning models
• Counterfactual Fairness
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Explaining Image Classifiers by Counterfactual Generation
• Noise or Signal: The Role of Image Backgrounds in Object Recognition
• GradMask: Reduce Overfitting by Regularizing Saliency
• Saliency Learning: Teaching the Model Where to Pay Attention