20 citations · 88 across the 10 of their papers we have counts for
10 papers
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
Zhengfu He, Tianxiang Sun, Kuanning Wang +2
We present DiffusionBERT, a new generative masked language model based on discrete diffusion models. Diffusion models and many pre-trained language models have a shared training ob…
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts
Xiangyang Liu, Tianxiang Sun, Xuanjing Huang +1
Prompt tuning is a parameter-efficient tuning (PETuning) method for utilizing pre-trained models (PTMs) that simply prepends a soft prompt to the input and only optimizes the promp…
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Tianxiang Sun, Junliang He, Xipeng Qiu +1
Automatic evaluation metrics are crucial to the development of generative systems. In recent years, pre-trained language model (PLM) based metrics, such as BERTScore, have been com…
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Tianxiang Sun, Xiangyang Liu, Wei Zhu +7
Early exiting allows instances to exit at different layers according to the estimation of difficulty. Previous works usually adopt heuristic metrics such as the entropy of internal…
Learning to Teach with Student Feedback
Yitao Liu, Tianxiang Sun, Xipeng Qiu +1
Knowledge distillation (KD) has gained much attention due to its effectiveness in compressing large-scale pre-trained models. In typical KD methods, the small student model is trai…
Accelerating BERT Inference for Sequence Labeling via Early-Exit
Xiaonan Li, Yunfan Shao, Tianxiang Sun +3
Both performance and efficiency are crucial factors for sequence labeling tasks in many real-world scenarios. Although the pre-trained models (PTMs) have significantly improved the…