On Learning and Learned Data Representation by Capsule Networks

Abstract

In this work, we investigate the following: 1) how the routing affects the CapsNet model fitting; 2) how the representation using capsules helps discover global structures in data distribution, and; 3) how the learned data representation adapts and generalizes to new tasks. Our investigation yielded the results some of which have been mentioned in the original paper of CapsNet, they are: 1) the routing operation determines the certainty with which a layer of capsules pass information to the layer above and the appropriate level of certainty is related to the model fitness; 2) in a designed experiment using data with a known 2D structure, capsule representations enable a more meaningful 2D manifold embedding than neurons do in a standard convolutional neural network (CNN), and; 3) compared with neurons of the standard CNN, capsules of successive layers are less coupled and more adaptive to new data distribution.

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• How transferable are features in deep neural networks?
• Understanding Black-box Predictions via Influence Functions
• Visualizing and Understanding Recurrent Networks
• Understanding Neural Networks through Representation Erasure
• Investigating Capsule Networks with Dynamic Routing for Text Classification
• CapsNet comparative performance evaluation for image classification
• CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces
• Sparse Unsupervised Capsules Generalize Better
• DARCCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules
• Generalized Capsule Networks with Trainable Routing Procedure

Cited by in corpus (1)

• Capsule Neural Networks for Graph Classification using Explicit Tensorial Graph Representations