12 papers
Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Licheng Pan, Haocheng Yang, Haoxuan Li +7
Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM v…
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF
Licheng Pan, Haocheng Yang, Yuzhou Chen +7
Feedback noise is pervasive in preference data, arising from subjective human judgments, annotation mistakes, and imperfect LLM judges. Standard reward modeling trained on such fee…
Not All Criteria Matter: Importance-Aware Criteria Weighting for Rubric-Based LLM Evaluation
Haocheng Yang, Licheng Pan, Yiming Dong +7
Rubric-based evaluation assesses the responses of large language models (LLMs) by aggregating the verification results of explicit criteria, yet not all criteria matter equally. Ho…
RubricArmor: Adversarial Evolution Improves LLM-Based Rubric Generation
Haocheng Yang, Yuchao Zhang, Licheng Pan +8
Rubric-based reinforcement learning (RL) provides interpretable rewards for aligning large language models (LLMs) by evaluating responses against query-specific evaluation criteria…
Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence
Haocheng Yang, Licheng Pan, Xiaoxi Li +8
Rubric evolution offers a promising approach to improving the quality of rubrics generated by large language models (LLMs). Central to this process is rubric comparison, which iden…
Unlocking In-Context Learning in Audio-Language Models from Decentralized Medical Audio
Ran Piao, Tsai-Ning Wang, Martijn den Dekker +4
Clinical audio diagnosis in low-resource settings requires models that identify conditions from minimal examples without large annotated corpora. We propose Federated Self-Contextu…
Uncertainty-Aware Reward Modeling for Stable RLHF
Licheng Pan, Haocheng Yang, Haoxuan Li +7
Reinforcement learning from human feedback (RLHF) aligns large language models by training reward models on preference data and optimizing policies to maximize predicted rewards. H…
Evaluating Chinese Ambiguity Understanding in Large Language Models
Junwen Mo, Yuanzhi Lu, Yifang Xue +2
Linguistic ambiguity is critical to the robustness of Large Language Models (LLMs), yet existing research focuses mostly on English, with limited attention devoted to Chinese. Exis…
Optimal Transport for LLM Reward Modeling from Noisy Preference
Licheng Pan, Haochen Yang, Haoxuan Li +8
Reward models are fundamental to Reinforcement Learning from Human Feedback (RLHF), yet real-world datasets are inevitably corrupted by noisy preference. Conventional training obje…
Unbiased Reward Modeling from Implicit Feedback for LLM Alignment
Hao Wang, Haocheng Yang, Licheng Pan +7
Despite the success of reinforcement learning from human feedback (RLHF), existing reward modeling methods largely rely on explicit feedback, which is costly to collect and difficu…
CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbacks
Hao Wang, Licheng Pan, Zhichao Chen +7
Despite the success of reinforcement learning from human feedback (RLHF) in aligning language models, current reward modeling heavily relies on experimental feedback data collected…
Rewards as Labels: Revisiting RLVR from a Classification Perspective
Zepeng Zhai, Meilin Chen, Jiaxuan Zhao +3
Reinforcement Learning with Verifiable Rewards has recently advanced the capabilities of Large Language Models in complex reasoning tasks by providing explicit rule-based supervisi…