1.4k citations · 5.4k across the 120 of their papers we have counts for
28 papers · 1 filter
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
Yujia Qin, Yankai Lin, Ryuichi Takanobu +6
Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks. However, conventional pre-training objectives do n…
Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
Si Sun, Yingzhuo Qian, Zhenghao Liu +5
The effectiveness of Neural Information Retrieval (Neu-IR) often depends on a large scale of in-domain relevance training signals, which are not always available in real-world rank…
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Zhengyan Zhang, Xu Han, Hao Zhou +22
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot o…
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
Hao Peng, Tianyu Gao, Xu Han +5
Neural models have achieved remarkable success on relation extraction (RE) benchmarks. However, there is no clear understanding which type of information affects existing RE models…
Denoising Relation Extraction from Document-level Distant Supervision
Chaojun Xiao, Yuan Yao, Ruobing Xie +5
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance. However, the existing succes…
Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu +2
Deep pre-trained Transformer models have achieved state-of-the-art results over a variety of natural language processing (NLP) tasks. By learning rich language knowledge with milli…
CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search
Chenyan Xiong, Zhenghao Liu, Si Sun +7
Neural rankers based on deep pretrained language models (LMs) have been shown to improve many information retrieval benchmarks. However, these methods are affected by their the cor…
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Fanchao Qi, Yangyi Chen, Mukai Li +3
Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of…
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph
Xin Lv, Xu Han, Lei Hou +6
Multi-hop reasoning has been widely studied in recent years to seek an effective and interpretable method for knowledge graph (KG) completion. Most previous reasoning methods are d…
Partially-Aligned Data-to-Text Generation with Distant Supervision
Zihao Fu, Bei Shi, Wai Lam +2
The Data-to-Text task aims to generate human-readable text for describing some given structured data enabling more interpretability. However, the typical generation task is confine…
Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect
Zheni Zeng, Chaojun Xiao, Yuan Yao +5
Recommender systems aim to provide item recommendations for users, and are usually faced with data sparsity problem (e.g., cold start) in real-world scenarios. Recently pre-trained…
Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations
Yuan Zang, Bairu Hou, Fanchao Qi +3
Adversarial attacking aims to fool deep neural networks with adversarial examples. In the field of natural language processing, various textual adversarial attack models have been…
Country Image in COVID-19 Pandemic: A Case Study of China
Huimin Chen, Zeyu Zhu, Fanchao Qi +4
Country image has a profound influence on international relations and economic development. In the worldwide outbreak of COVID-19, countries and their people display different reac…
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
Yusheng Su, Xu Han, Zhengyan Zhang +5
Several recent efforts have been devoted to enhancing pre-trained language models (PLMs) by utilizing extra heterogeneous knowledge in knowledge graphs (KGs) and achieved consisten…
OpenAttack: An Open-source Textual Adversarial Attack Toolkit
Guoyang Zeng, Fanchao Qi, Qianrui Zhou +6
Textual adversarial attacking has received wide and increasing attention in recent years. Various attack models have been proposed, which are enormously distinct and implemented wi…
Adaptive Graph Encoder for Attributed Graph Embedding
Ganqu Cui, Jie Zhou, Cheng Yang +1
Attributed graph embedding, which learns vector representations from graph topology and node features, is a challenging task for graph analysis. Recently, methods based on graph co…
Few-Shot Generative Conversational Query Rewriting
Shi Yu, Jiahua Liu, Jingqin Yang +4
Conversational query rewriting aims to reformulate a concise conversational query to a fully specified, context-independent query that can be effectively handled by existing inform…
Crossmodal Language Grounding in an Embodied Neurocognitive Model
Stefan Heinrich, Yuan Yao, Tobias Hinz +5
Human infants are able to acquire natural language seemingly easily at an early age. Their language learning seems to occur simultaneously with learning other cognitive functions a…
Graph Policy Network for Transferable Active Learning on Graphs
Shengding Hu, Zheng Xiong, Meng Qu +4
Graph neural networks (GNNs) have been attracting increasing popularity due to their simplicity and effectiveness in a variety of fields. However, a large number of labeled data is…
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
Haoxi Zhong, Chaojun Xiao, Cunchao Tu +3
Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain.…
Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen
Yixin Cao, Ruihao Shui, Liangming Pan +3
The curse of knowledge can impede communication between experts and laymen. We propose a new task of expertise style transfer and contribute a manually annotated dataset with the g…
KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion
Jie Zhou, Shengding Hu, Xin Lv +6
A comprehensive knowledge graph (KG) contains an instance-level entity graph and an ontology-level concept graph. The two-view KG provides a testbed for models to "simulate" human'…
MAVEN: A Massive General Domain Event Detection Dataset
Xiaozhi Wang, Ziqi Wang, Xu Han +7
Event detection (ED), which means identifying event trigger words and classifying event types, is the first and most fundamental step for extracting event knowledge from plain text…
Train No Evil: Selective Masking for Task-Guided Pre-Training
Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang +2
Recently, pre-trained language models mostly follow the pre-train-then-fine-tuning paradigm and have achieved great performance on various downstream tasks. However, since the pre-…