Self-Adaptive Hierarchical Sentence Model
arXiv:1504.05070
Abstract
The ability to accurately model a sentence at varying stages (e.g., word-phrase-sentence) plays a central role in natural language processing. As an effort towards this goal we propose a self-adaptive hierarchical sentence model (AdaSent). AdaSent effectively forms a hierarchy of representations from words to phrases and then to sentences through recursive gated local composition of adjacent segments. We design a competitive mechanism (through gating networks) to allow the representations of the same sentence to be engaged in a particular learning task (e.g., classification), therefore effectively mitigating the gradient vanishing problem persistent in other recursive models. Both qualitative and quantitative analysis shows that AdaSent can automatically form and select the representations suitable for the task at hand during training, yielding superior classification performance over competitor models on 5 benchmark data sets.
8 pages, 7 figures, accepted as a full paper at IJCAI 2015
References in corpus (5)
- Distributed Representations of Words and Phrases and their Compositionality
- Natural Language Processing (almost) from Scratch
- Distributed Representations of Sentences and Documents
- On the difficulty of training Recurrent Neural Networks
- Convolutional Neural Network Architectures for Matching Natural Language Sentences
Cited by in corpus (10)
- Skip-Thought Vectors
- Adversarial Training Methods for Semi-Supervised Text Classification
- Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
- SeMantic AnsweR Type prediction task (SMART) at ISWC 2020 Semantic Web Challenge
- Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation
- Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Study
- Depth-Adaptive Graph Recurrent Network for Text Classification
- Multi-Granular Text Encoding for Self-Explaining Categorization
- On the difficulty of a distributional semantics of spoken language
- Interpretable Structure-aware Document Encoders with Hierarchical Attention