86 citations · 232 across the 11 of their papers we have counts for
10 papers · 1 filter
Improving Language Model Reasoning with Self-motivated Learning
Yunlong Feng, Yang Xu, Libo Qin +2
Large-scale high-quality training data is important for improving the performance of models. After trained with data that has rationales (reasoning steps), models gain reasoning ca…
Momentum Contrastive Pre-training for Question Answering
Minda Hu, Muzhi Li, Yasheng Wang +1
Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-train…
Sparse Structure Search for Parameter-Efficient Tuning
Shengding Hu, Zhen Zhang, Ning Ding +4
Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of parameter-efficient tuning (PET) find that onl…
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Fanchao Qi, Mukai Li, Yangyi Chen +4
Backdoor attacks are a kind of insidious security threat against machine learning models. After being injected with a backdoor in training, the victim model will produce adversary-…
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li +6
Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, w…
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
Chenglei Si, Zhengyan Zhang, Fanchao Qi +4
Pretrained language models (PLMs) perform poorly under adversarial attacks. To improve the adversarial robustness, adversarial data augmentation (ADA) has been widely adopted to co…
Multi-channel Reverse Dictionary Model
Lei Zhang, Fanchao Qi, Zhiyuan Liu +3
A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description. Existing reverse dictionary m…
Improving Sequence Modeling Ability of Recurrent Neural Networks via Sememes
Yujia Qin, Fanchao Qi, Sicong Ouyang +5
Sememes, the minimum semantic units of human languages, have been successfully utilized in various natural language processing applications. However, most existing studies exploit…
NEZHA: Neural Contextualized Representation for Chinese Language Understanding
Junqiu Wei, Xiaozhe Ren, Xiaoguang Li +7
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized informati…
GPT-based Generation for Classical Chinese Poetry
Yi Liao, Yasheng Wang, Qun Liu +1
We present a simple yet effective method for generating high quality classical Chinese poetry with Generative Pre-trained Language Model (GPT). The method adopts a simple GPT model…