Do CNNs Encode Data Augmentations?

Abstract

Data augmentations are important ingredients in the recipe for training robust neural networks, especially in computer vision. A fundamental question is whether neural network features encode data augmentation transformations. To answer this question, we introduce a systematic approach to investigate which layers of neural networks are the most predictive of augmentation transformations. Our approach uses features in pre-trained vision models with minimal additional processing to predict common properties transformed by augmentation (scale, aspect ratio, hue, saturation, contrast, and brightness). Surprisingly, neural network features not only predict data augmentation transformations, but they predict many transformations with high accuracy. After validating that neural networks encode features corresponding to augmentation transformations, we show that these features are encoded in the early layers of modern CNNs, though the augmentation signal fades in deeper layers.

References in corpus (16)

• mixup: Beyond Empirical Risk Minimization
• Neural Architecture Search with Reinforcement Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• Understanding deep learning requires rethinking generalization
• Group Equivariant Convolutional Networks
• Making Convolutional Networks Shift-Invariant Again
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Self-training with Noisy Student improves ImageNet classification
• Locally Scale-Invariant Convolutional Neural Networks
• Scale-Invariant Convolutional Neural Networks
• A Kernel Theory of Modern Data Augmentation
• How Much Position Information Do Convolutional Neural Networks Encode?
• Adversarial Examples Improve Image Recognition
• Scale-Equivariant Steerable Networks
• Radioactive data: tracing through training
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters