Paradigm Shift in Natural Language Processing
arXiv:2109.12575 · doi:10.1007/s11633-022-1331-6
Abstract
In the era of deep learning, modeling for most NLP tasks has converged to several mainstream paradigms. For example, we usually adopt the sequence labeling paradigm to solve a bundle of tasks such as POS-tagging, NER, Chunking, and adopt the classification paradigm to solve tasks like sentiment analysis. With the rapid progress of pre-trained language models, recent years have observed a rising trend of Paradigm Shift, which is solving one NLP task by reformulating it as another one. Paradigm shift has achieved great success on many tasks, becoming a promising way to improve model performance. Moreover, some of these paradigms have shown great potential to unify a large number of NLP tasks, making it possible to build a single model to handle diverse tasks. In this paper, we review such phenomenon of paradigm shifts in recent years, highlighting several paradigms that have the potential to solve different NLP tasks.
arXiv commentAccepted to Machine Intelligence Research. Published version: https://link.springer.com/article/10.1007/s11633-022-1331-6. Website: https://txsun1997.github.io/nlp-paradigm-shift
References in corpus (35)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Sequence to Sequence Learning with Neural Networks
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Convolutional Sequence to Sequence Learning
- Bidirectional Attention Flow for Machine Comprehension
- Pre-trained Models for Natural Language Processing: A Survey
- Enhanced LSTM for Natural Language Inference
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- Transition-Based Dependency Parsing with Stack Long Short-Term Memory
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Non-Autoregressive Neural Machine Translation
- Dynamic Coattention Networks For Question Answering
- Grammar as a Foreign Language
- The Natural Language Decathlon: Multitask Learning as Question Answering
- Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
- Entailment as Few-Shot Learner
- Multi-Grained Named Entity Recognition
- Improving Neural Language Models with a Continuous Cache
- Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification
- Depth-Adaptive Transformer
- SentiPrompt: Sentiment Knowledge Enhanced Prompt-Tuning for Aspect-Based Sentiment Analysis
- SGM: Sequence Generation Model for Multi-label Classification
- Structured Prediction as Translation between Augmented Natural Languages
- HTLM: Hyper-Text Pre-Training and Prompting of Language Models
- Description Based Text Classification with Reinforcement Learning
- NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction
- A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis
- PTR: Prompt Tuning with Rules for Text Classification
- BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
- Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
- Factual Probing Is [MASK]: Learning vs. Learning to Recall
- Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
- Read, Retrospect, Select: An MRC Framework to Short Text Entity Linking
- Dependency Parsing as MRC-based Span-Span Prediction