Integrating Approaches to Word Representation
arXiv:2109.04876
Abstract
The problem of representing the atomic elements of language in modern neural learning systems is one of the central challenges of the field of natural language processing. I present a survey of the distributional, compositional, and relational approaches to addressing this task, and discuss various means of integrating them into systems, with special emphasis on the word level and the out-of-vocabulary phenomenon.
arXiv commentAdapted dissertation introduction
References in corpus (13)
- Efficient Estimation of Word Representations in Vector Space
- Natural Language Processing (almost) from Scratch
- Recent Trends in Deep Learning Based Natural Language Processing
- Character-Aware Neural Language Models
- Global Organization of the Lexicon
- ProjE: Embedding Projection for Knowledge Graph Completion
- CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation
- Modeling Relational Data with Graph Convolutional Networks
- CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters
- What to do about non-standard (or non-canonical) language in NLP
- How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?
- Learning to Look Inside: Augmenting Token-Based Encoders with Character-Level Information
- Attending Form and Context to Generate Specialized Out-of-VocabularyWords Representations