1.4k citations · 5.4k across the 120 of their papers we have counts for
122 papers
QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation
Kunlun Zhu, Shihao Liang, Xu Han +4
Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks. However, obtaining sufficie…
LLM As DBA
Xuanhe Zhou, Guoliang Li, Zhiyuan Liu
Database administrators (DBAs) play a crucial role in managing, maintaining and optimizing a database system to ensure data availability, performance, and reliability. However, it…
Automatic Truss Design with Reinforcement Learning
Weihua Du, Jinglun Zhao, Chao Yu +7
Truss layout design, namely finding a lightweight truss layout satisfying all the physical constraints, is a fundamental problem in the building industry. Generating the optimal la…
GACT: Activation Compressed Training for Generic Network Architectures
Xiaoxuan Liu, Lianmin Zheng, Dequan Wang +9
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.…
From Symbols to Embeddings: A Tale of Two Representations in Computational Social Science
Huimin Chen, Cheng Yang, Xuanming Zhang +3
Computational Social Science (CSS), aiming at utilizing computational methods to address social science problems, is a recent emerging and fast-developing field. The study of CSS i…
CPM-2: Large-scale Cost-effective Pre-trained Language Models
Zhengyan Zhang, Yuxian Gu, Xu Han +16
In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-…
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
Fanchao Qi, Yuan Yao, Sophia Xu +2
Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but…
Automatic Construction of Sememe Knowledge Bases via Dictionaries
Fanchao Qi, Yangyi Chen, Fengyu Wang +3
A sememe is defined as the minimum semantic unit in linguistics. Sememe knowledge bases (SKBs), which comprise words annotated with sememes, enable sememes to be applied to natural…
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Fanchao Qi, Mukai Li, Yangyi Chen +4
Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-…
Evaluating Modules in Graph Contrastive Learning
Ganqu Cui, Yufeng Du, Cheng Yang +5
The recent emergence of contrastive learning approaches facilitates the application on graph representation learning (GRL), introducing graph contrastive learning (GCL) into the li…
Pre-Trained Models: Past, Present and Future
Xu Han, Zhengyan Zhang, Ning Ding +21
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophis…
CLEVE: Contrastive Pre-training for Event Extraction
Ziqi Wang, Xiaozhi Wang, Xu Han +6
Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning. However, existing pre-training methods have not involved modeling event cha…
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Tianyu Gao, Xu Han, Keyue Qiu +7
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data. However, its evaluation has l…
Few-Shot Conversational Dense Retrieval
Shi Yu, Zhenghao Liu, Chenyan Xiong +2
Dense retrieval (DR) has the potential to resolve the query understanding challenge in conversational search by matching in the learned embedding space. However, this adaptation is…
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents
Chaojun Xiao, Xueyu Hu, Zhiyuan Liu +2
Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP). Recently, inspir…
Knowledge Inheritance for Pre-trained Language Models
Yujia Qin, Yankai Lin, Jing Yi +8
Recent explorations of large-scale pre-trained language models (PLMs) have revealed the power of PLMs with huge amounts of parameters, setting off a wave of training ever-larger PL…
PTR: Prompt Tuning with Rules for Text Classification
Xu Han, Weilin Zhao, Ning Ding +2
Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the…
Few-NERD: A Few-Shot Named Entity Recognition Dataset
Ning Ding, Guangwei Xu, Yulin Chen +5
Recently, considerable literature has grown up around the theme of few-shot named entity recognition (NER), but little published benchmark data specifically focused on the practica…
Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability
Xin Lv, Yixin Cao, Lei Hou +4
Multi-hop reasoning has been widely studied in recent years to obtain more interpretable link prediction. However, we find in experiments that many paths given by these models are…
Equality before the Law: Legal Judgment Consistency Analysis for Fairness
Yuzhong Wang, Chaojun Xiao, Shirong Ma +5
In a legal system, judgment consistency is regarded as one of the most important manifestations of fairness. However, due to the complexity of factual elements that impact sentenci…
Visual Distant Supervision for Scene Graph Generation
Yuan Yao, Ao Zhang, Xu Han +5
Scene graph generation aims to identify objects and their relations in images, providing structured image representations that can facilitate numerous applications in computer visi…
Representation Learning for Natural Language Processing
Zhiyuan Liu, Yankai Lin, Maosong Sun
This book aims to review and present the recent advances of distributed representation learning for NLP, including why representation learning can improve NLP, how representation l…
OpenMatch: An Open Source Library for Neu-IR Research
Zhenghao Liu, Kaitao Zhang, Chenyan Xiong +2
OpenMatch is a Python-based library that serves for Neural Information Retrieval (Neu-IR) research. It provides self-contained neural and traditional IR modules, making it easy to…
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li +6
Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, w…