Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
arXiv:1808.08561
Abstract
We propose a novel model for multi-label text classification, which is based on sequence-to-sequence learning. The model generates higher-level semantic unit representations with multi-level dilated convolution as well as a corresponding hybrid attention mechanism that extracts both the information at the word-level and the level of the semantic unit. Our designed dilated convolution effectively reduces dimension and supports an exponential expansion of receptive fields without loss of local information, and the attention-over-attention mechanism is able to capture more summary relevant information from the source context. Results of our experiments show that the proposed model has significant advantages over the baseline models on the dataset RCV1-V2 and Ren-CECps, and our analysis demonstrates that our model is competitive to the deterministic hierarchical models and it is more robust to classifying low-frequency labels.
arXiv commentEMNLP 2018
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- WaveNet: A Generative Model for Raw Audio
- Convolutional Sequence to Sequence Learning
- Multi-Scale Context Aggregation by Dilated Convolutions
- Effective Approaches to Attention-based Neural Machine Translation
- A Convolutional Neural Network for Modelling Sentences
- Neural Machine Translation in Linear Time
- Sememe Prediction: Learning Semantic Knowledge from Unstructured Textual Wiki Descriptions