7 citations · 16 across the 17 of their papers we have counts for
14 papers · 1 filter
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
Yifu Huo, Chenglong Wang, Ziming Zhu +9
Reinforcement learning (RL) has emerged as a promising paradigm for training reasoning-oriented models by leveraging rule-based reward signals. However, RL training typically tends…
Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
Chenglong Wang, Yifu Huo, Yang Gan +10
Previous methods evaluate reward models by testing them on a fixed pairwise ranking test set, but they typically do not provide performance information on each preference dimension…
MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization
Chenglong Wang, Yang Gan, Hang Zhou +10
Recent advances in diffusion language models (DLMs) have presented a promising alternative to traditional autoregressive large language models (LLMs). However, DLMs still lag behin…
HEAL: A Hypothesis-Based Preference-Aware Analysis Framework
Yifu Huo, Chenglong Wang, Qiren Zhu +5
Preference optimization methods like DPO have achieved remarkable performance in LLM alignment. However, the evaluation for these methods relies on a single response and overlooks…
GRAM: A Generative Foundation Reward Model for Reward Generalization
Chenglong Wang, Yang Gan, Yifu Huo +8
In aligning large language models (LLMs), reward models have played an important role, but are standardly trained as discriminative models and rely only on labeled human preference…
LRHP: Learning Representations for Human Preferences via Preference Pairs
Chenglong Wang, Yang Gan, Yifu Huo +7
To improve human-preference alignment training, current research has developed numerous preference datasets consisting of preference pairs labeled as "preferred" or "dispreferred".…
Hybrid Alignment Training for Large Language Models
Chenglong Wang, Hang Zhou, Kaiyan Chang +5
Alignment training is crucial for enabling large language models (LLMs) to cater to human intentions and preferences. It is typically performed based on two stages with different o…
Revealing the Parallel Multilingual Learning within Large Language Models
Yongyu Mu, Peinan Feng, Zhiquan Cao +8
In this study, we reveal an in-context learning (ICL) capability of multilingual large language models (LLMs): by translating the input to several languages, we provide Parallel In…
ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation
Chenglong Wang, Hang Zhou, Yimin Hu +5
Applying Reinforcement Learning (RL) to sequence generation models enables the direct optimization of long-term rewards (\textit{e.g.,} BLEU and human feedback), but typically requ…
Learning Evaluation Models from Large Language Models for Sequence Generation
Chenglong Wang, Hang Zhou, Kaiyan Chang +6
Automatic evaluation of sequence generation, traditionally reliant on metrics like BLEU and ROUGE, often fails to capture the semantic accuracy of generated text sequences due to t…
Towards Fully 8-bit Integer Inference for the Transformer Model
Ye Lin, Yanyang Li, Tengbo Liu +3
8-bit integer inference, as a promising direction in reducing both the latency and storage of deep neural networks, has made great progress recently. On the other hand, previous sy…
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation
Bei Li, Hui Liu, Ziyang Wang +5
In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence. In this paper, we investigat…
Neural Machine Translation with Joint Representation
Yanyang Li, Qiang Wang, Tong Xiao +2
Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units,…
Sharing Attention Weights for Fast Transformer
Tong Xiao, Yinqiao Li, Jingbo Zhu +2
Recently, the Transformer machine translation system has shown strong results by stacking attention layers on both the source and target-language sides. But the inference of this m…