58 citations · 372 across the 25 of their papers we have counts for
6 papers · 1 filter
Everyone Deserves A Reward: Learning Customized Human Preferences
Pengyu Cheng, Jiawen Xie, Ke Bai +2
Reward models (RMs) are essential for aligning large language models (LLMs) with human preferences to improve interaction quality. However, the real world is pluralistic, which lea…
Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers
Jiawen Xie, Pengyu Cheng, Xiao Liang +2
Although dominant in natural language processing, transformer-based models remain challenged by the task of long-sequence processing, because the computational cost of self-attenti…
Toward Fairness in Text Generation via Mutual Information Minimization based on Importance Sampling
Rui Wang, Pengyu Cheng, Ricardo Henao
Pretrained language models (PLMs), such as GPT2, have achieved remarkable empirical performance in text generation tasks. However, pretrained on large-scale natural language corpor…
FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
Pengyu Cheng, Weituo Hao, Siyang Yuan +2
Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance ga…
Learning Compressed Sentence Representations for On-Device Text Processing
Dinghan Shen, Pengyu Cheng, Dhanasekar Sundararaman +5
Vector representations of sentences, trained on massive text corpora, are widely used as generic sentence embeddings across a variety of NLP problems. The learned representations a…
Improving Textual Network Embedding with Global Attention via Optimal Transport
Liqun Chen, Guoyin Wang, Chenyang Tao +6
Constituting highly informative network embeddings is an important tool for network analysis. It encodes network topology, along with other useful side information, into low-dimens…