Learning Augmentation Distributions using Transformed Risk Minimization

Abstract

We propose a new \emph{Transformed Risk Minimization} (TRM) framework as an extension of classical risk minimization. In TRM, we optimize not only over predictive models, but also over data transformations; specifically over distributions thereof. As a key application, we focus on learning augmentations; for instance appropriate rotations of images, to improve classification performance with a given class of predictors. Our TRM method (1) jointly learns transformations and models in a \emph{single training loop}, (2) works with any training algorithm applicable to standard risk minimization, and (3) handles any transforms, such as discrete and continuous classes of augmentations. To avoid overfitting when implementing empirical transformed risk minimization, we propose a novel regularizer based on PAC-Bayes theory. For learning augmentations of images, we propose a new parametrization of the space of augmentations via a stochastic composition of blocks of geometric transforms. This leads to the new \emph{Stochastic Compositional Augmentation Learning} (SCALE) algorithm. The performance of TRM with SCALE compares favorably to prior methods on CIFAR10/100. Additionally, we show empirically that SCALE can correctly learn certain symmetries in the data distribution (recovering rotations on rotated MNIST) and can also improve calibration of the learned model.

References in corpus (27)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Categorical Reparameterization with Gumbel-Softmax
• Recent Advances in Physical Reservoir Computing: A Review
• On Calibration of Modern Neural Networks
• Data Augmentation Generative Adversarial Networks
• Random Erasing Data Augmentation
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Deep Anomaly Detection Using Geometric Transformations
• Generalization Properties of Learning with Random Features
• General $E(2)$-Equivariant Steerable CNNs
• Classification-Based Anomaly Detection for General Data
• Invariant and Equivariant Graph Networks
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• A Group-Theoretic Framework for Data Augmentation
• Sketched Ridge Regression: Optimization Perspective, Statistical Perspective, and Model Averaging
• A More Powerful Two-Sample Test in High Dimensions using Random Projection
• Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience
• Graph Normalizing Flows
• Learning Invariances using the Marginal Likelihood
• Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares
• Asymptotics for Sketching in Least Squares Regression
• Meta-Learning Symmetries by Reparameterization
• Generalization Bounds for Vicinal Risk Minimization Principle
• The Nyström method for convex loss functions