100 citations · 205 across the 25 of their papers we have counts for
7 papers · 1 filter
Self-Prompting Large Language Models for Zero-Shot Open-Domain QA
Junlong Li, Jinyuan Wang, Zhuosheng Zhang +1
Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing specific background documents. This task becomes notably challenging in a zero-shot sett…
Language Model Pre-training on True Negatives
Zhuosheng Zhang, Hai Zhao, Masao Utiyama +1
Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Taking the former text as positive and the latter as negative s…
Instance Regularization for Discriminative Language Model Pre-training
Zhuosheng Zhang, Hai Zhao, Ming Zhou
Discriminative pre-trained language models (PrLMs) can be generalized as denoising auto-encoders that work with two procedures, ennoising and denoising. First, an ennoising process…
Semantic-Preserving Adversarial Code Comprehension
Yiyang Li, Hongqiu Wu, Hai Zhao
Based on the tremendous success of pre-trained language models (PrLMs) for source code comprehension tasks, current literature studies either ways to further improve the performanc…
Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning
Letian Peng, Zuchao Li, Hai Zhao
Commonsense reasoning is an appealing topic in natural language processing (NLP) as it plays a fundamental role in supporting the human-like actions of NLP systems. With large-scal…
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang, Zhuosheng Zhang, Hai Zhao
Masked Language Modeling (MLM) has been widely used as the denoising objective in pre-training language models (PrLMs). Existing PrLMs commonly adopt a Random-Token Masking strateg…
Rethinking Textual Adversarial Defense for Pre-trained Language Models
Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang +1
Although pre-trained language models (PrLMs) have achieved significant success, recent studies demonstrate that PrLMs are vulnerable to adversarial attacks. By generating adversari…