Investigating Capsule Networks with Dynamic Routing for Text Classification

Abstract

In this study, we explore capsule networks with dynamic routing for text classification. We propose three strategies to stabilize the dynamic routing process to alleviate the disturbance of some noise capsules which may contain "background" information or have not been successfully trained. A series of experiments are conducted with capsule networks on six text classification benchmarks. Capsule networks achieve state of the art on 4 out of 6 datasets, which shows the effectiveness of capsule networks for text classification. We additionally show that capsule networks exhibit significant improvement when transfer single-label to multi-label text classification over strong baseline methods. To the best of our knowledge, this is the first work that capsule networks have been empirically investigated for text modeling.

12 pages

References in corpus (6)

• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Supervised Topic Models
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• Capsule Network Performance on Complex Data
• Linguistically Regularized LSTMs for Sentiment Classification
• Fast Dynamic Routing Based on Weighted Kernel Density Estimation

Cited by in corpus (19)

• Deep Learning Based Text Classification: A Comprehensive Review
• A Survey on Text Classification: From Shallow to Deep Learning
• Text Classification using Capsules
• Joint Slot Filling and Intent Detection via Capsule Neural Networks
• SECaps: A Sequence Enhanced Capsule Model for Charge Prediction
• Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction
• On Learning and Learned Data Representation by Capsule Networks
• Self-attention-based BiGRU and capsule network for named entity recognition
• Sentiment Analysis for Sinhala Language using Deep Learning Techniques
• Dynamic Past and Future for Neural Machine Translation
• Multi-modal Capsule Routing for Actor and Action Video Segmentation Conditioned on Natural Language Queries
• A Capsule Network for Recommendation and Explaining What You Like and Dislike
• Capsule network with shortcut routing
• TreeCaps: Tree-Structured Capsule Networks for Program Source Code Processing
• Explainable Recommender Systems via Resolving Learning Representations
• TraceCaps: A Capsule-based Neural Network for Semantic Segmentation
• GRET: Global Representation Enhanced Transformer
• Graph Routing between Capsules
• An Efficient Agreement Mechanism in CapsNets By Pairwise Product