Style Augmentation: Data Augmentation via Style Randomization

Abstract

We introduce style augmentation, a new form of data augmentation based on random style transfer, for improving the robustness of convolutional neural networks (CNN) over both classification and regression based tasks. During training, our style augmentation randomizes texture, contrast and color, while preserving shape and semantic content. This is accomplished by adapting an arbitrary style transfer network to perform style randomization, by sampling input style embeddings from a multivariate normal distribution instead of inferring them from a style image. In addition to standard classification experiments, we investigate the effect of style augmentation (and data augmentation generally) on domain transfer tasks. We find that data augmentation significantly improves robustness to domain shift, and can be used as a simple, domain agnostic alternative to domain adaptation. Comparing style augmentation against a mix of seven traditional augmentation techniques, we find that it can be readily combined with them to improve network performance. We validate the efficacy of our technique with domain transfer experiments in classification and monocular depth estimation, illustrating consistent improvements in generalization.

Cited by in corpus (18)

• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• An overview of mixing augmentation methods and augmentation strategies
• Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation
• UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models
• Frustratingly Simple Domain Generalization via Image Stylization
• MISS GAN: A Multi-IlluStrator Style Generative Adversarial Network for image to illustration translation
• Rethinking Recurrent Neural Networks and Other Improvements for Image Classification
• An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
• SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap
• Class balanced underwater object detection dataset generated by class-wise style augmentation
• VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
• Restyling Data: Application to Unsupervised Domain Adaptation
• Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for Better Single-Source Domain Generalization
• Data Augmentation via Mixed Class Interpolation using Cycle-Consistent Generative Adversarial Networks Applied to Cross-Domain Imagery
• Cross-Domain Image Classification through Neural-Style Transfer Data Augmentation
• Non-Parametric Neural Style Transfer
• Octave Mix: Data augmentation using frequency decomposition for activity recognition
• The Role of the Input in Natural Language Video Description