5 papers
Simple Denoising Diffusion Language Models
Huaisheng Zhu, Zhengyu Chen, Shijie Zhou +8
Recent Uniform State Diffusion Models (USDMs), initialized from a uniform prior, offer the promise of fast text generation due to their inherent self-correction ability compared to…
Discovering Invariant Neighborhood Patterns for Heterophilic Graphs
Jinluan Yang, Ruihao Zhang, Zhengyu Chen +4
This paper studies the problem of distribution shifts on non-homophilous graphs Mosting existing graph neural network methods rely on the homophilous assumption that nodes from the…
On a Connection Between Imitation Learning and RLHF
Teng Xiao, Yige Yuan, Mingxiao Li +2
This work studies the alignment of large language models with preference data from an imitation learning perspective. We establish a close theoretical connection between reinforcem…
Leveraging Invariant Principle for Heterophilic Graph Structure Distribution Shifts
Jinluan Yang, Zhengyu Chen, Teng Xiao +3
Heterophilic Graph Neural Networks (HGNNs) have shown promising results for semi-supervised learning tasks on graphs. Notably, most real-world heterophilic graphs are composed of a…
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters
Teng Xiao, Yige Yuan, Zhengyu Chen +4
Existing preference optimization objectives for language model alignment require additional hyperparameters that must be extensively tuned to achieve optimal performance, increasin…