DropBlock: A regularization method for convolutional networks
arXiv:1810.12890
Abstract
Deep neural networks often work well when they are over-parameterized and trained with a massive amount of noise and regularization, such as weight decay and dropout. Although dropout is widely used as a regularization technique for fully connected layers, it is often less effective for convolutional layers. This lack of success of dropout for convolutional layers is perhaps due to the fact that activation units in convolutional layers are spatially correlated so information can still flow through convolutional networks despite dropout. Thus a structured form of dropout is needed to regularize convolutional networks. In this paper, we introduce DropBlock, a form of structured dropout, where units in a contiguous region of a feature map are dropped together. We found that applying DropbBlock in skip connections in addition to the convolution layers increases the accuracy. Also, gradually increasing number of dropped units during training leads to better accuracy and more robust to hyperparameter choices. Extensive experiments show that DropBlock works better than dropout in regularizing convolutional networks. On ImageNet classification, ResNet-50 architecture with DropBlock achieves accuracy, which is more than improvement on the baseline. On COCO detection, DropBlock improves Average Precision of RetinaNet from to .
arXiv commentAccepted at NIPS 2018
Cited by in corpus (157)
- YOLOv4: Optimal Speed and Accuracy of Object Detection
- A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS
- CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
- Learning Spatial Fusion for Single-Shot Object Detection
- Rethinking Pre-training and Self-training
- PP-YOLO: An Effective and Efficient Implementation of Object Detector
- GridMask Data Augmentation
- Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification
- Revisiting ResNets: Improved Training and Scaling Strategies
- NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
- Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
- Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
- Adaptive DropBlock Enhanced Generative Adversarial Networks for Hyperspectral Image Classification
- Review: Deep Learning in Electron Microscopy
- Second-order Non-local Attention Networks for Person Re-identification
- Aerial Images Processing for Car Detection using Convolutional Neural Networks: Comparison between Faster R-CNN and YoloV3
- Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
- SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization
- Incomplete Descriptor Mining with Elastic Loss for Person Re-Identification
- Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
- An overview of mixing augmentation methods and augmentation strategies
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
- Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions
- Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence
- In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
- Effective and Efficient Dropout for Deep Convolutional Neural Networks
- Multi-Sample Dropout for Accelerated Training and Better Generalization
- MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks
- MSAF: Multimodal Split Attention Fusion
- Self-Challenging Improves Cross-Domain Generalization
- Confidence Calibration for Convolutional Neural Networks Using Structured Dropout
- Domain Generalization with MixStyle
- ResizeMix: Mixing Data with Preserved Object Information and True Labels
- A global method to identify trees outside of closed-canopy forests with medium-resolution satellite imagery
- SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation
- LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor
- Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network
- Cut-Thumbnail: A Novel Data Augmentation for Convolutional Neural Network
- An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
- MaxUp: A Simple Way to Improve Generalization of Neural Network Training
- DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
- PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks
- Global Second-order Pooling Convolutional Networks
- Optimized Generic Feature Learning for Few-shot Classification across Domains
- On Feature Normalization and Data Augmentation
- Rethinking "Batch" in BatchNorm
- Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection
- GradAug: A New Regularization Method for Deep Neural Networks
- Cross-Domain Few-Shot Learning by Representation Fusion
- Batch DropBlock Network for Person Re-identification and Beyond
- Task Augmentation by Rotating for Meta-Learning
- SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization
- Mish: A Self Regularized Non-Monotonic Activation Function
- How to Train Your MAML to Excel in Few-Shot Classification
- Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks
- Learning ordered pooling weights in image classification
- MixStyle Neural Networks for Domain Generalization and Adaptation
- Temporal Complementary Learning for Video Person Re-Identification
- On Episodes, Prototypical Networks, and Few-shot Learning
- Revisiting 3D ResNets for Video Recognition
- When Does Self-supervision Improve Few-shot Learning?
- LibFewShot: A Comprehensive Library for Few-shot Learning
- KeepAugment: A Simple Information-Preserving Data Augmentation Approach
- Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet
- Temperature check: theory and practice for training models with softmax-cross-entropy losses
- An Empirical Analysis of the Impact of Data Augmentation on Knowledge Distillation
- LocalDrop: A Hybrid Regularization for Deep Neural Networks
- Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning
- A deep learning-based framework for segmenting invisible clinical target volumes with estimated uncertainties for post-operative prostate cancer radiotherapy
- Proposal Learning for Semi-Supervised Object Detection
- Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation
- Region Comparison Network for Interpretable Few-shot Image Classification
- Neural Architecture Search for Deep Face Recognition
- Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
- Deep Learning in Diabetic Foot Ulcers Detection: A Comprehensive Evaluation
- Revisiting the Sibling Head in Object Detector
- Instance Credibility Inference for Few-Shot Learning
- Adversarially Adaptive Normalization for Single Domain Generalization
- Min-max Entropy for Weakly Supervised Pointwise Localization
- Improving Neural Network with Uniform Sparse Connectivity
- Scale-aware Automatic Augmentation for Object Detection
- In the Eye of the Beholder: Gaze and Actions in First Person Video
- Regularizing activations in neural networks via distribution matching with the Wasserstein metric
- Adaptive Label Smoothing
- Feature Pyramid Transformer
- DropCluster: A structured dropout for convolutional networks
- Rethinking Training from Scratch for Object Detection
- Improving image classifiers for small datasets by learning rate adaptations
- Regularizing Meta-Learning via Gradient Dropout
- Residual Spatial Attention Network for Retinal Vessel Segmentation
- Scheduled DropHead: A Regularization Method for Transformer Models
- Convolutional Neural Networks with Dynamic Regularization
- HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization
- Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
- Channel Attention Residual U-Net for Retinal Vessel Segmentation
- Repaint: Improving the Generalization of Down-Stream Visual Tasks by Generating Multiple Instances of Training Examples
- Vehicle Re-ID for Surround-view Camera System
- Inner-Imaging Networks: Put Lenses into Convolutional Structure
- Multi-Loss Sub-Ensembles for Accurate Classification with Uncertainty Estimation
- Meta Dropout: Learning to Perturb Features for Generalization
- Learning Temporally Invariant and Localizable Features via Data Augmentation for Video Recognition
- Partial Graph Reasoning for Neural Network Regularization
- Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics
- Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
- Branch-Cooperative OSNet for Person Re-Identification
- Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes
- How to trust unlabeled data? Instance Credibility Inference for Few-Shot Learning
- Representation Sharing for Fast Object Detector Search and Beyond
- MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures
- SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation
- An ETF view of Dropout regularization
- Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives
- ResiliNet: Failure-Resilient Inference in Distributed Neural Networks
- Incremental Meta-Learning via Indirect Discriminant Alignment
- High-order structure preserving graph neural network for few-shot learning
- Defective Convolutional Networks
- On the Demystification of Knowledge Distillation: A Residual Network Perspective
- Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks
- DropFilter: A Novel Regularization Method for Learning Convolutional Neural Networks
- A simple and effective postprocessing method for image classification
- Shot in the Dark: Few-Shot Learning with No Base-Class Labels
- Dual-attention Guided Dropblock Module for Weakly Supervised Object Localization
- EENA: Efficient Evolution of Neural Architecture
- Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation
- Progressive Multi-stage Feature Mix for Person Re-Identification
- Explainability-aided Domain Generalization for Image Classification
- UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
- Augmentation Inside the Network
- Improving compute efficacy frontiers with SliceOut
- Don't ignore Dropout in Fully Convolutional Networks
- Modeling Artistic Workflows for Image Generation and Editing
- Extended Few-Shot Learning: Exploiting Existing Resources for Novel Tasks
- Exploiting Token and Path-based Representations of Code for Identifying Security-Relevant Commits
- Data Augmentation via Mixed Class Interpolation using Cycle-Consistent Generative Adversarial Networks Applied to Cross-Domain Imagery
- Channel DropBlock: An Improved Regularization Method for Fine-Grained Visual Classification
- Sequential Feature Filtering Classifier
- Decoupled Box Proposal and Featurization with Ultrafine-Grained Semantic Labels Improve Image Captioning and Visual Question Answering
- Improve SGD Training via Aligning Mini-batches
- Intra-Model Collaborative Learning of Neural Networks
- ShuffleBlock: Shuffle to Regularize Deep Convolutional Neural Networks
- Robust Learning with Frequency Domain Regularization
- Push for Center Learning via Orthogonalization and Subspace Masking for Person Re-Identification
- Selective Output Smoothing Regularization: Regularize Neural Networks by Softening Output Distributions
- Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight
- Balanced Masked and Standard Face Recognition
- Predicting Driver Intention Using Deep Neural Network
- Uniform Sampling over Episode Difficulty
- Granular Motor State Monitoring of Free Living Parkinson's Disease Patients via Deep Learning
- Robust Regularization with Adversarial Labelling of Perturbed Samples
- Mutually-aware Sub-Graphs Differentiable Architecture Search
- Credal Self-Supervised Learning
- Jitter: Random Jittering Loss Function
- Detecting and Segmenting Adversarial Graphics Patterns from Images
- Dense Residual Network for Retinal Vessel Segmentation
- RotationOut as a Regularization Method for Neural Network
- Stochastic Region Pooling: Make Attention More Expressive
- Go Small and Similar: A Simple Output Decay Brings Better Performance