Sparse Unsupervised Capsules Generalize Better
arXiv:1804.06094
Abstract
We show that unsupervised training of latent capsule layers using only the reconstruction loss, without masking to select the correct output class, causes a loss of equivariances and other desirable capsule qualities. This implies that supervised capsules networks can't be very deep. Unsupervised sparsening of latent capsule layer activity both restores these qualities and appears to generalize better than supervised masking, while potentially enabling deeper capsules networks. We train a sparse, unsupervised capsules network of similar geometry to Sabour et al (2017) on MNIST, and then test classification accuracy on affNIST using an SVM layer. Accuracy is improved from benchmark 79% to 90%.
References in corpus (5)
Cited by in corpus (18)
- Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks
- How Can We Be So Dense? The Benefits of Using Highly Sparse Representations
- HitNet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules
- Stacked Capsule Autoencoders
- Equivariant Transformer Networks
- Generalized Capsule Networks with Trainable Routing Procedure
- Capsule Networks Need an Improved Routing Algorithm
- Unsupervised part representation by Flow Capsules
- On Learning and Learned Data Representation by Capsule Networks
- Effective and Efficient Vote Attack on Capsule Networks
- Building Deep, Equivariant Capsule Networks
- Examining the Benefits of Capsule Neural Networks
- Improving the Robustness of Capsule Networks to Image Affine Transformations
- Capsule Network is Not More Robust than Convolutional Network
- An Effective Hit-or-Miss Layer Favoring Feature Interpretation as Learned Prototypes Deformations
- Grouping Capsules Based Different Types
- Capsule Networks -- A Probabilistic Perspective
- Capsule Networks Do Not Need to Model Everything