Supervised Deep Sparse Coding Networks
arXiv:1701.08349
Abstract
In this paper, we describe the deep sparse coding network (SCN), a novel deep network that encodes intermediate representations with nonnegative sparse coding. The SCN is built upon a number of cascading bottleneck modules, where each module consists of two sparse coding layers with relatively wide and slim dictionaries that are specialized to produce high dimensional discriminative features and low dimensional representations for clustering, respectively. During training, both the dictionaries and regularization parameters are optimized with an end-to-end supervised learning algorithm based on multilevel optimization. Effectiveness of an SCN with seven bottleneck modules is verified on several popular benchmark datasets. Remarkably, with few parameters to learn, our SCN achieves 5.81% and 19.93% classification error rate on CIFAR-10 and CIFAR-100, respectively.
References in corpus (24)
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Rejoinder to "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Least Angle Regression
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Discussion of "Least angle regression" by Efron et al
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Improving neural networks by preventing co-adaptation of feature detectors
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Robust Recovery of Subspace Structures by Low-Rank Representation
- Striving for Simplicity: The All Convolutional Net
- Wide Residual Networks
- Task-Driven Dictionary Learning
- Highway Networks
- Aggregated Residual Transformations for Deep Neural Networks
- ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
- Stacked What-Where Auto-encoders
- Swapout: Learning an ensemble of deep architectures
- Understanding Trainable Sparse Coding via Matrix Factorization
- Deep TEN: Texture Encoding Network