Sentence-State LSTM for Text Representation
arXiv:1805.02474
Abstract
Bi-directional LSTMs are a powerful tool for text representation. On the other hand, they have been shown to suffer various limitations due to their sequential nature. We investigate an alternative LSTM structure for encoding text, which consists of a parallel state for each word. Recurrent steps are used to perform local and global information exchange between words simultaneously, rather than incremental reading of a sequence of words. Results on various classification and sequence labelling benchmarks show that the proposed model has strong representation power, giving highly competitive performances compared to stacked BiLSTM models with similar parameter numbers.
arXiv commentACL 18 camera-ready version
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Teaching Machines to Read and Comprehend
- Loopy Belief Propagation for Approximate Inference: An Empirical Study
- Transition-Based Dependency Parsing with Stack Long Short-Term Memory
- A Convolutional Neural Network for Modelling Sentences
- LSTM-based Deep Learning Models for Non-factoid Answer Selection
- Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
- Linguistically Regularized LSTMs for Sentiment Classification
Cited by in corpus (17)
- Graph Convolutional Networks for Text Classification
- Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model
- Word Mover's Embedding: From Word2Vec to Document Embedding
- Hierarchical Contextualized Representation for Named Entity Recognition
- VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
- Recursive Graphical Neural Networks for Text Classification
- Graph-based Neural Sentence Ordering
- Jointly Modeling Aspect and Sentiment with Dynamic Heterogeneous Graph Neural Networks
- Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
- Dual-Attention Graph Convolutional Network
- Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP
- Better Feature Integration for Named Entity Recognition
- KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers
- BERT-hLSTMs: BERT and Hierarchical LSTMs for Visual Storytelling
- Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization
- Dynamic Graph Embedding via LSTM History Tracking
- GRET: Global Representation Enhanced Transformer