Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data
arXiv:1802.00003 · doi:10.1109/TNNLS.2017.2747861
Abstract
Unsupervised feature extractors are known to perform an efficient and discriminative representation of data. Insight into the mappings they perform and human ability to understand them, however, remain very limited. This is especially prominent when multilayer deep learning architectures are used. This paper demonstrates how to remove these bottlenecks within the architecture of Nonnegativity Constrained Autoencoder (NCSAE). It is shown that by using both L1 and L2 regularization that induce nonnegativity of weights, most of the weights in the network become constrained to be nonnegative thereby resulting into a more understandable structure with minute deterioration in classification accuracy. Also, this proposed approach extracts features that are more sparse and produces additional output layer sparsification. The method is analyzed for accuracy and feature interpretation on the MNIST data, the NORB normalized uniform object data, and the Reuters text categorization dataset.
arXiv commentIEEE Trans. on Neural Networks and Learning Systems, September 2018, Vol. 29, Issue 9, Pg. 3969 - 3979
References in corpus (4)
Cited by in corpus (8)
- Knowledge Adaptation for Efficient Semantic Segmentation
- Multiplicative update rules for accelerating deep learning training and increasing robustness
- Exploiting the Full Capacity of Deep Neural Networks while Avoiding Overfitting by Targeted Sparsity Regularization
- Neural Text Classification by Jointly Learning to Cluster and Align
- Learning a Deep Part-based Representation by Preserving Data Distribution
- Diversity Regularized Adversarial Learning
- Part-based approximations for morphological operators using asymmetric auto-encoders
- The Impact of Activation Sparsity on Overfitting in Convolutional Neural Networks