1.4k citations · 5.4k across the 120 of their papers we have counts for
13 papers · 2 filters
CPM-2: Large-scale Cost-effective Pre-trained Language Models
Zhengyan Zhang, Yuxian Gu, Xu Han +16
In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-…
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
Fanchao Qi, Yuan Yao, Sophia Xu +2
Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but…
Automatic Construction of Sememe Knowledge Bases via Dictionaries
Fanchao Qi, Yangyi Chen, Fengyu Wang +3
A sememe is defined as the minimum semantic unit in linguistics. Sememe knowledge bases (SKBs), which comprise words annotated with sememes, enable sememes to be applied to natural…
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Fanchao Qi, Mukai Li, Yangyi Chen +4
Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-…
CLEVE: Contrastive Pre-training for Event Extraction
Ziqi Wang, Xiaozhi Wang, Xu Han +6
Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning. However, existing pre-training methods have not involved modeling event cha…
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Tianyu Gao, Xu Han, Keyue Qiu +7
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data. However, its evaluation has l…
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents
Chaojun Xiao, Xueyu Hu, Zhiyuan Liu +2
Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP). Recently, inspir…
Knowledge Inheritance for Pre-trained Language Models
Yujia Qin, Yankai Lin, Jing Yi +8
Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PL…
PTR: Prompt Tuning with Rules for Text Classification
Xu Han, Weilin Zhao, Ning Ding +2
Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the…
Few-NERD: A Few-Shot Named Entity Recognition Dataset
Ning Ding, Guangwei Xu, Yulin Chen +5
Recently, considerable literature has grown up around the theme of few-shot named entity recognition (NER), but little published benchmark data specifically focused on the practica…
Equality before the Law: Legal Judgment Consistency Analysis for Fairness
Yuzhong Wang, Chaojun Xiao, Shirong Ma +5
In a legal system, judgment consistency is regarded as one of the most important manifestations of fairness. However, due to the complexity of factual elements that impact sentenci…
Representation Learning for Natural Language Processing
Zhiyuan Liu, Yankai Lin, Maosong Sun
This book aims to review and present the recent advances of distributed representation learning for NLP, including why representation learning can improve NLP, how representation l…
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li +6
Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, w…