SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization

Abstract

Advanced data augmentation strategies have widely been studied to improve the generalization ability of deep learning models. Regional dropout is one of the popular solutions that guides the model to focus on less discriminative parts by randomly removing image regions, resulting in improved regularization. However, such information removal is undesirable. On the other hand, recent strategies suggest to randomly cut and mix patches and their labels among training images, to enjoy the advantages of regional dropout without having any pointless pixel in the augmented images. We argue that such random selection strategies of the patches may not necessarily represent sufficient information about the corresponding object and thereby mixing the labels according to that uninformative patch enables the model to learn unexpected feature representation. Therefore, we propose SaliencyMix that carefully selects a representative image patch with the help of a saliency map and mixes this indicative patch with the target image, thus leading the model to learn more appropriate feature representation. SaliencyMix achieves the best known top-1 error of 21.26% and 20.09% for ResNet-50 and ResNet-101 architectures on ImageNet classification, respectively, and also improves the model robustness against adversarial perturbations. Furthermore, models that are trained with SaliencyMix help to improve the object detection performance. Source code is available at https://github.com/SaliencyMix/SaliencyMix.

12 pages, 5 figures, 5 tables

References in corpus (2)

• Improved Regularization of Convolutional Neural Networks with Cutout
• Deep Image: Scaling up Image Recognition

Cited by in corpus (13)

• Automated data processing and feature engineering for deep learning and big data applications: a survey
• FMix: Enhancing Mixed Sample Data Augmentation
• An overview of mixing augmentation methods and augmentation strategies
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• ResizeMix: Mixing Data with Preserved Object Information and True Labels
• GuidedMixup: An Efficient Mixup Strategy Guided by Saliency Maps
• It Takes Two to Tango: Mixup for Deep Metric Learning
• Catch-Up Mix: Catch-Up Class for Struggling Filters in CNN
• Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability
• SuperpixelGridCut, SuperpixelGridMean and SuperpixelGridMix Data Augmentation
• AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer
• SMCL: Saliency Masked Contrastive Learning for Long-tailed Recognition
• Adversarial AutoMixup