Supervised Deep Sparse Coding Networks
arXiv:1701.08349
Abstract
In this paper, we describe the deep sparse coding network (SCN), a novel deep network that encodes intermediate representations with nonnegative sparse coding. The SCN is built upon a number of cascading bottleneck modules, where each module consists of two sparse coding layers with relatively wide and slim dictionaries that are specialized to produce high dimensional discriminative features and low dimensional representations for clustering, respectively. During training, both the dictionaries and regularization parameters are optimized with an end-to-end supervised learning algorithm based on multilevel optimization. Effectiveness of an SCN with seven bottleneck modules is verified on several popular benchmark datasets. Remarkably, with few parameters to learn, our SCN achieves 5.81% and 19.93% classification error rate on CIFAR-10 and CIFAR-100, respectively.
References in corpus (7)
- Improving neural networks by preventing co-adaptation of feature detectors
- Striving for Simplicity: The All Convolutional Net
- Wide Residual Networks
- Aggregated Residual Transformations for Deep Neural Networks
- Swapout: Learning an ensemble of deep architectures
- Understanding Trainable Sparse Coding via Matrix Factorization
- Deep TEN: Texture Encoding Network