1.4k citations · 5.4k across the 120 of their papers we have counts for
19 papers · 2 filters
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
Yujia Qin, Yankai Lin, Ryuichi Takanobu +6
Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks. However, conventional pre-training objectives do n…
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Zhengyan Zhang, Xu Han, Hao Zhou +22
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot o…
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
Hao Peng, Tianyu Gao, Xu Han +5
Neural models have achieved remarkable success on relation extraction (RE) benchmarks. However, there is no clear understanding which type of information affects existing RE models…
Denoising Relation Extraction from Document-level Distant Supervision
Chaojun Xiao, Yuan Yao, Ruobing Xie +5
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance. However, the existing succes…
Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu +2
Deep pre-trained Transformer models have achieved state-of-the-art results over a variety of natural language processing (NLP) tasks. By learning rich language knowledge with milli…
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Fanchao Qi, Yangyi Chen, Mukai Li +3
Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of…
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph
Xin Lv, Xu Han, Lei Hou +6
Multi-hop reasoning has been widely studied in recent years to seek an effective and interpretable method for knowledge graph (KG) completion. Most previous reasoning methods are d…
Partially-Aligned Data-to-Text Generation with Distant Supervision
Zihao Fu, Bei Shi, Wai Lam +2
The Data-to-Text task aims to generate human-readable text for describing some given structured data enabling more interpretability. However, the typical generation task is confine…
Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations
Yuan Zang, Bairu Hou, Fanchao Qi +3
Adversarial attacking aims to fool deep neural networks with adversarial examples. In the field of natural language processing, various textual adversarial attack models have been…
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
Yusheng Su, Xu Han, Zhengyan Zhang +5
Several recent efforts have been devoted to enhancing pre-trained language models (PLMs) by utilizing extra heterogeneous knowledge in knowledge graphs (KGs) and achieved consisten…
OpenAttack: An Open-source Textual Adversarial Attack Toolkit
Guoyang Zeng, Fanchao Qi, Qianrui Zhou +6
Textual adversarial attacking has received wide and increasing attention in recent years. Various attack models have been proposed, which are enormously distinct and implemented wi…
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
Haoxi Zhong, Chaojun Xiao, Cunchao Tu +3
Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain.…
Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen
Yixin Cao, Ruihao Shui, Liangming Pan +3
The curse of knowledge can impede communication between experts and laymen. We propose a new task of expertise style transfer and contribute a manually annotated dataset with the g…
KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion
Jie Zhou, Shengding Hu, Xin Lv +6
A comprehensive knowledge graph (KG) contains an instance-level entity graph and an ontology-level concept graph. The two-view KG provides a testbed for models to "simulate" human'…
MAVEN: A Massive General Domain Event Detection Dataset
Xiaozhi Wang, Ziqi Wang, Xu Han +7
Event detection (ED), which means identifying event trigger words and classifying event types, is the first and most fundamental step for extracting event knowledge from plain text…
Train No Evil: Selective Masking for Task-Guided Pre-Training
Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang +2
Recently, pre-trained language models mostly follow the pre-train-then-fine-tuning paradigm and have achieved great performance on various downstream tasks. However, since the pre-…
Coreferential Reasoning Learning for Language Representation
Deming Ye, Yankai Lin, Jiaju Du +4
Language representation models such as BERT could effectively capture contextual semantic information from plain text, and have been proved to achieve promising results in lots of…
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Xu Han, Tianyu Gao, Yankai Lin +7
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text. In order to extract these facts from text, people have been working on rel…
Lexical Sememe Prediction using Dictionary Definitions by Capturing Local Semantic Correspondence
Jiaju Du, Fanchao Qi, Maosong Sun +1
Sememes, defined as the minimum semantic units of human languages in linguistics, have been proven useful in many NLP tasks. Since manual construction and update of sememe knowledg…