P-CapsNets: a General Form of Convolutional Neural Networks
arXiv:1912.08367
Abstract
We propose Pure CapsNets (P-CapsNets) which is a generation of normal CNNs structurally. Specifically, we make three modifications to current CapsNets. First, we remove routing procedures from CapsNets based on the observation that the coupling coefficients can be learned implicitly. Second, we replace the convolutional layers in CapsNets to improve efficiency. Third, we package the capsules into rank-3 tensors to further improve efficiency. The experiment shows that P-CapsNets achieve better performance than CapsNets with varied routing procedures by using significantly fewer parameters on MNIST\&CIFAR10. The high efficiency of P-CapsNets is even comparable to some deep compressing models. For example, we achieve more than 99\% percent accuracy on MNIST by using only 3888 parameters. We visualize the capsules as well as the corresponding correlation matrix to show a possible way of initializing CapsNets in the future. We also explore the adversarial robustness of P-CapsNets compared to CNNs.
References in corpus (11)
- Explaining and Harnessing Adversarial Examples
- Caffe: Convolutional Architecture for Fast Feature Embedding
- cuDNN: Efficient Primitives for Deep Learning
- Capsule Network Performance on Complex Data
- Universal adversarial perturbations
- Dense and Diverse Capsule Networks: Making the Capsules Learn Better
- Stacked Capsule Autoencoders
- Generalized Capsule Networks with Trainable Routing Procedure
- Capsule Networks Need an Improved Routing Algorithm
- ProdSumNet: reducing model parameters in deep neural networks via product-of-sums matrix decompositions
- Fast Dynamic Routing Based on Weighted Kernel Density Estimation