18 citations · 23 across the 38 of their papers we have counts for
8 papers · 1 filter
LRHP: Learning Representations for Human Preferences via Preference Pairs
Chenglong Wang, Yang Gan, Yifu Huo +7
To improve human-preference alignment training, current research has developed numerous preference datasets consisting of preference pairs labeled as "preferred" or "dispreferred".…
NDP: Next Distribution Prediction as a More Broad Target
Junhao Ruan, Abudukeyumu Abudula, Xinyu Liu +7
Large language models (LLMs) trained on next-token prediction (NTP) paradigm have demonstrated powerful capabilities. However, the existing NTP paradigm contains several limitation…
RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
Chenglong Wang, Yang Gan, Yifu Huo +9
Large vision-language models (LVLMs) often fail to align with human preferences, leading to issues like generating misleading content without proper visual context (also known as h…
Cross-layer Attention Sharing for Pre-trained Large Language Models
Yongyu Mu, Yuzhang Wu, Yuchun Fan +9
To enhance the efficiency of the attention mechanism within large language models (LLMs), previous works primarily compress the KV cache or group attention heads, while largely ove…
Hybrid Alignment Training for Large Language Models
Chenglong Wang, Hang Zhou, Kaiyan Chang +5
Alignment training is crucial for enabling large language models (LLMs) to cater to human intentions and preferences. It is typically performed based on two stages with different o…
Efficient Prompting Methods for Large Language Models: A Survey
Kaiyan Chang, Songcheng Xu, Chenglong Wang +4
Prompting is a mainstream paradigm for adapting large language models to specific natural language processing tasks without modifying internal parameters. Therefore, detailed suppl…
Prior Constraints-based Reward Model Training for Aligning Large Language Models
Hang Zhou, Chenglong Wang, Yimin Hu +3
Reinforcement learning with human feedback for aligning large language models (LLMs) trains a reward model typically using ranking loss with comparison pairs.However, the training…
Revealing the Parallel Multilingual Learning within Large Language Models
Yongyu Mu, Peinan Feng, Zhiquan Cao +8
In this study, we reveal an in-context learning (ICL) capability of multilingual large language models (LLMs): by translating the input to several languages, we provide Parallel In…