Survey of Dropout Methods for Deep Neural Networks
arXiv:1904.13310
Abstract
Dropout methods are a family of stochastic techniques used in neural network training or inference that have generated significant research interest and are widely used in practice. They have been successfully applied in neural network regularization, model compression, and in measuring the uncertainty of neural network outputs. While original formulated for dense neural network layers, recent advances have made dropout methods also applicable to convolutional and recurrent neural network layers. This paper summarizes the history of dropout methods, their various applications, and current areas of research interest. Important proposed methods are described in additional detail.
References in corpus (10)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Improving neural networks by preventing co-adaptation of feature detectors
- Improved Regularization of Convolutional Neural Networks with Cutout
- Recurrent Neural Network Regularization
- Regularizing and Optimizing LSTM Language Models
- Deep and Confident Prediction for Time Series at Uber
- Dropout Inference in Bayesian Neural Networks with Alpha-divergences
- Effective and Efficient Dropout for Deep Convolutional Neural Networks
- Ising-Dropout: A Regularization Method for Training and Compression of Deep Neural Networks
- Adversarial Dropout for Recurrent Neural Networks
Cited by in corpus (11)
- R-Drop: Regularized Dropout for Neural Networks
- Review: Deep Learning in Electron Microscopy
- Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization
- EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks
- HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization
- EventDrop: data augmentation for event-based learning
- Dual-attention Guided Dropblock Module for Weakly Supervised Object Localization
- Noisy-LSTM: Improving Temporal Awareness for Video Semantic Segmentation
- Locally Learned Synaptic Dropout for Complete Bayesian Inference
- Quantal synaptic dilution enhances sparse encoding and dropout regularisation in deep networks
- RotationOut as a Regularization Method for Neural Network