237 citations · 1.2k across the 87 of their papers we have counts for
11 papers · 1 filter
Self-Prompting Large Language Models for Zero-Shot Open-Domain QA
Junlong Li, Jinyuan Wang, Zhuosheng Zhang +1
Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing specific background documents. This task becomes notably challenging in a zero-shot sett…
Language Model Pre-training on True Negatives
Zhuosheng Zhang, Hai Zhao, Masao Utiyama +1
Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Taking the former text as positive and the latter as negative s…
Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
Wenhao Yu, Chenguang Zhu, Zhihan Zhang +4
A common thread of retrieval-augmented methods in the existing literature focuses on retrieving encyclopedic knowledge, such as Wikipedia, which facilitates well-defined entity and…
Task Compass: Scaling Multi-task Pre-training with Task Prefix
Zhuosheng Zhang, Shuohang Wang, Yichong Xu +6
Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language mod…
Instance Regularization for Discriminative Language Model Pre-training
Zhuosheng Zhang, Hai Zhao, Ming Zhou
Discriminative pre-trained language models (PrLMs) can be generalized as denoising auto-encoders that work with two procedures, ennoising and denoising. First, an ennoising process…
Automatic Chain of Thought Prompting in Large Language Models
Zhuosheng Zhang, Aston Zhang, Mu Li +1
Large language models (LLMs) can perform complex reasoning by generating intermediate reasoning steps. Providing these steps for prompting demonstrations is called chain-of-thought…
Towards End-to-End Open Conversational Machine Reading
Sizhe Zhou, Siru Ouyang, Zhuosheng Zhang +1
In open-retrieval conversational machine reading (OR-CMR) task, machines are required to do multi-turn question answering given dialogue history and a textual knowledge base. Exist…
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang, Zhuosheng Zhang, Hai Zhao
Masked Language Modeling (MLM) has been widely used as the denoising objective in pre-training language models (PrLMs). Existing PrLMs commonly adopt a Random-Token Masking strateg…
Rethinking Textual Adversarial Defense for Pre-trained Language Models
Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang +1
Although pre-trained language models (PrLMs) have achieved significant success, recent studies demonstrate that PrLMs are vulnerable to adversarial attacks. By generating adversari…
Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling
Yiyang Li, Hai Zhao, Zhuosheng Zhang
Multi-turn dialogue modeling as a challenging branch of natural language understanding (NLU), aims to build representations for machines to understand human dialogues, which provid…
Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model
Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang +1
Recently, the problem of robustness of pre-trained language models (PrLMs) has received increasing research interest. Latest studies on adversarial attacks achieve high attack succ…