Paradigm Shift in Natural Language Processing

Abstract

In the era of deep learning, modeling for most NLP tasks has converged to several mainstream paradigms. For example, we usually adopt the sequence labeling paradigm to solve a bundle of tasks such as POS-tagging, NER, Chunking, and adopt the classification paradigm to solve tasks like sentiment analysis. With the rapid progress of pre-trained language models, recent years have observed a rising trend of Paradigm Shift, which is solving one NLP task by reformulating it as another one. Paradigm shift has achieved great success on many tasks, becoming a promising way to improve model performance. Moreover, some of these paradigms have shown great potential to unify a large number of NLP tasks, making it possible to build a single model to handle diverse tasks. In this paper, we review such phenomenon of paradigm shifts in recent years, highlighting several paradigms that have the potential to solve different NLP tasks.

Accepted to Machine Intelligence Research. Published version: https://link.springer.com/article/10.1007/s11633-022-1331-6. Website: https://txsun1997.github.io/nlp-paradigm-shift

References in corpus (20)

• Sequence to Sequence Learning with Neural Networks
• MASS: Masked Sequence to Sequence Pre-training for Language Generation
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
• Grammar as a Foreign Language
• Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
• Entailment as Few-Shot Learner
• Improving Neural Language Models with a Continuous Cache
• Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification
• Depth-Adaptive Transformer
• SentiPrompt: Sentiment Knowledge Enhanced Prompt-Tuning for Aspect-Based Sentiment Analysis
• HTLM: Hyper-Text Pre-Training and Prompting of Language Models
• NSP-BERT: A Prompt-based Few-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction
• A Joint Training Dual-MRC Framework for Aspect Based Sentiment Analysis
• PTR: Prompt Tuning with Rules for Text Classification
• Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
• Factual Probing Is [MASK]: Learning vs. Learning to Recall
• Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
• Read, Retrospect, Select: An MRC Framework to Short Text Entity Linking
• Dependency Parsing as MRC-based Span-Span Prediction

Cited by in corpus (4)

• EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training
• Masked Vision-Language Transformer in Fashion
• The Life Cycle of Knowledge in Big Language Models: A Survey
• Analyzing FOMC Minutes: Accuracy and Constraints of Language Models