SelectScale: Mining More Patterns from Images via Selective and Soft Dropout

Abstract

Convolutional neural networks (CNNs) have achieved remarkable success in image recognition. Although the internal patterns of the input images are effectively learned by the CNNs, these patterns only constitute a small proportion of useful patterns contained in the input images. This can be attributed to the fact that the CNNs will stop learning if the learned patterns are enough to make a correct classification. Network regularization methods like dropout and SpatialDropout can ease this problem. During training, they randomly drop the features. These dropout methods, in essence, change the patterns learned by the networks, and in turn, forces the networks to learn other patterns to make the correct classification. However, the above methods have an important drawback. Randomly dropping features is generally inefficient and can introduce unnecessary noise. To tackle this problem, we propose SelectScale. Instead of randomly dropping units, SelectScale selects the important features in networks and adjusts them during training. Using SelectScale, we improve the performance of CNNs on CIFAR and ImageNet.

arXiv commentarXiv admin note: text overlap with arXiv:1810.09849 by other authors

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Improved Regularization of Convolutional Neural Networks with Cutout
• Densely Connected Convolutional Networks
• Wide Residual Networks
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Aggregated Residual Transformations for Deep Neural Networks
• Deep Networks with Stochastic Depth