Charagram: Embedding Words and Sentences via Character n-grams
arXiv:1607.02789
Abstract
We present Charagram embeddings, a simple approach for learning character-based compositional models to embed textual sequences. A word or sentence is represented using a character n-gram count vector, followed by a single nonlinear transformation to yield a low-dimensional embedding. We use three tasks for evaluation: word similarity, sentence similarity, and part-of-speech tagging. We demonstrate that Charagram embeddings outperform more complex architectures based on character-level recurrent and convolutional neural networks, achieving new state-of-the-art performance on several similarity tasks.
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Exploring the Limits of Language Modeling
- A large annotated corpus for learning natural language inference
- Character-based Neural Machine Translation
- Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
- Embedding Word Similarity with Neural Machine Translation
- Non-distributional Word Vector Representations
Cited by in corpus (20)
- Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
- SentEval: An Evaluation Toolkit for Universal Sentence Representations
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
- A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks
- Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
- Long Short-Term Memory for Japanese Word Segmentation
- Flow-Loss: Learning Cardinality Estimates That Matter
- Neural Paraphrase Identification of Questions with Noisy Pretraining
- Chinese Embedding via Stroke and Glyph Information: A Dual-channel View
- Sentence Modeling via Multiple Word Embeddings and Multi-level Comparison for Semantic Textual Similarity
- Incorporating Chinese Characters of Words for Lexical Sememe Prediction
- Improving Pre-Trained Multilingual Models with Vocabulary Expansion
- Terminology-based Text Embedding for Computing Document Similarities on Technical Content
- Neural Machine Translation with Source-Side Latent Graph Parsing
- Nonsymbolic Text Representation
- VCWE: Visual Character-Enhanced Word Embeddings
- Authorship Attribution in Bangla literature using Character-level CNN
- Automated Word Stress Detection in Russian
- Improving Distributed Representations of Tweets - Present and Future
- Patterns versus Characters in Subword-aware Neural Language Modeling