Deep Feature Response Discriminative Calibration

Abstract

Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach. The developed code is publicly available at https://github.com/tcmyxc/ResCNet.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Auto-Encoding Variational Bayes
• mixup: Beyond Empirical Risk Minimization
• Gaussian Error Linear Units (GELUs)
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Improved Regularization of Convolutional Neural Networks with Cutout
• Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification
• Symbolic Discovery of Optimization Algorithms