ConAM: Confidence Attention Module for Convolutional Neural Networks

Abstract

The so-called "attention" is an efficient mechanism to improve the performance of convolutional neural networks. It uses contextual information to recalibrate the input to strengthen the propagation of informative features. However, the majority of the attention mechanisms only consider either local or global contextual information, which is singular to extract features. Moreover, many existing mechanisms directly use the contextual information to recalibrate the input, which unilaterally enhances the propagation of the informative features, but does not suppress the useless ones. This paper proposes a new attention mechanism module based on the correlation between local and global contextual information and we name this correlation as confidence. The novel attention mechanism extracts the local and global contextual information simultaneously, and calculates the confidence between them, then uses this confidence to recalibrate the input pixels. The extraction of local and global contextual information increases the diversity of features. The recalibration with confidence suppresses useless information while enhancing the informative one with fewer parameters. We use CIFAR-10 and CIFAR-100 in our experiments and explore the performance of our method's components by sufficient ablation studies. Finally, we compare our method with a various state-of-the-art convolutional neural networks and the results show that our method completely surpasses these models. We implement ConAM with the Python library, Pytorch, and the code and models will be publicly available.

We need to withdraw this article due to a conflict of interest

References in corpus (10)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Neural Architecture Search with Reinforcement Learning
• Improved Regularization of Convolutional Neural Networks with Cutout
• Striving for Simplicity: The All Convolutional Net
• Automatically designing CNN architectures using genetic algorithm for image classification
• A Survey on Evolutionary Neural Architecture Search
• Large-Scale Evolution of Image Classifiers
• Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey
• Efficient Architecture Search by Network Transformation