Boosting Network Weight Separability via Feed-Backward Reconstruction

Abstract

This paper proposes a new evaluation metric and boosting method for weight separability in neural network design. In contrast to general visual recognition methods designed to encourage both intra-class compactness and inter-class separability of latent features, we focus on estimating linear independence of column vectors in weight matrix and improving the separability of weight vectors. To this end, we propose an evaluation metric for weight separability based on semi-orthogonality of a matrix and Frobenius distance, and the feed-backward reconstruction loss which explicitly encourages weight separability between the column vectors in the weight matrix. The experimental results on image classification and face recognition demonstrate that the weight separability boosting via minimization of feed-backward reconstruction loss can improve the visual recognition performance, hence universally boosting the performance on various visual recognition tasks.

arXiv comment8 pages, 6 figures

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Res2Net: A New Multi-scale Backbone Architecture
• Training Very Deep Networks
• DeepID3: Face Recognition with Very Deep Neural Networks
• AutoAugment: Learning Augmentation Policies from Data
• Drivers Drowsiness Detection using Condition-Adaptive Representation Learning Framework
• Regularizing CNNs with Locally Constrained Decorrelations
• Relaxed 2-D Principal Component Analysis by $L_p$ Norm for Face Recognition