PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks

Abstract

Large capacity deep learning models are often prone to a high generalization gap when trained with a limited amount of labeled training data. A recent class of methods to address this problem uses various ways to construct a new training sample by mixing a pair (or more) of training samples. We propose PatchUp, a hidden state block-level regularization technique for Convolutional Neural Networks (CNNs), that is applied on selected contiguous blocks of feature maps from a random pair of samples. Our approach improves the robustness of CNN models against the manifold intrusion problem that may occur in other state-of-the-art mixing approaches. Moreover, since we are mixing the contiguous block of features in the hidden space, which has more dimensions than the input space, we obtain more diverse samples for training towards different dimensions. Our experiments on CIFAR10/100, SVHN, Tiny-ImageNet, and ImageNet using ResNet architectures including PreActResnet18/34, WRN-28-10, ResNet101/152 models show that PatchUp improves upon, or equals, the performance of current state-of-the-art regularizers for CNNs. We also show that PatchUp can provide a better generalization to deformed samples and is more robust against adversarial attacks.

arXiv commentAAAI - 2022

References in corpus (14)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Sequence to Sequence Learning with Neural Networks
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• mixup: Beyond Empirical Risk Minimization
• Improved Regularization of Convolutional Neural Networks with Cutout
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• DropBlock: A regularization method for convolutional networks
• Manifold Mixup: Better Representations by Interpolating Hidden States
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• A Closer Look at Memorization in Deep Networks
• Foolbox: A Python toolbox to benchmark the robustness of machine learning models
• MetaMixUp: Learning Adaptive Interpolation Policy of MixUp with Meta-Learning

Cited by in corpus (1)

• Local and Global Context-and-Object-part-Aware Superpixel-based Data Augmentation for Deep Visual Recognition