Machine Learning Meets Natural Language Processing -- The story so far
arXiv:2104.10213
Abstract
Natural Language Processing (NLP) has evolved significantly over the last decade. This paper highlights the most important milestones of this period while trying to pinpoint the contribution of each individual model and algorithm to the overall progress. Furthermore, it focuses on issues still remaining to be solved, emphasizing the groundbreaking proposals of Transformers, BERT, and all the similar attention-based models.
arXiv comment13 pages, 5 figures
References in corpus (10)
- Distributed Representations of Words and Phrases and their Compositionality
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Language Models are Few-Shot Learners
- Cross-lingual Language Model Pretraining
- Enhanced LSTM for Natural Language Inference
- Convolutional Neural Networks for Sentence Classification
- Comparative Study of CNN and RNN for Natural Language Processing
- ERNIE: Enhanced Representation through Knowledge Integration
- A Convolutional Neural Network for Modelling Sentences
- Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context