7 papers
Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation
Yongliang Miao, Shuang Liu, Yanguang Liu +2
On-policy distillation (OPD) uses teacher correction on student-generated responses. Full-vocabulary correction can provide important corrections even for tokens that the student a…
RewardExplainer: Learning Reward Model Explanations from Counterfactual Preference Feedback
Jingyi He, Nier Wu, Shuang Liu +3
Reward models (RMs) are a key component of large language model post-training, providing reward signals for subsequent reinforcement learning. However, conventional discriminative…
BiasReducer: Adaptive Bias Mitigation for Reward Models
Shuang Liu, Yongliang Miao, Yanguang Liu +2
Reward models score responses from large language models (LLMs) and guide LLM training toward human preferences. However, reward models can favor superficial attributes such as len…
Conditional Multi-Event Temporal Grounding in Long-Form Video
Yuanhao Zou, Arthad Kulkarni, Lucas Tonanez +12
Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositi…
HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models
Shuang Liu, Yuxuan Bo, Qiuyang Zhao +4
Reward models are central to large language model (LLM) alignment, but they remain vulnerable to reward hacking. To evaluate reward-model robustness, we introduce RewardHackBench c…
LLM Agents in Law: Taxonomy, Applications, and Challenges
Shuang Liu, Ruijia Zhang, Ruoyun Ma +6
Large language models (LLMs) have precipitated a dramatic improvement in the legal domain, yet the deployment of standalone models faces significant limitations regarding hallucina…
ContractEval: Benchmarking LLMs for Clause-Level Legal Risk Identification in Commercial Contracts
Shuang Liu, Zelong Li, Ruoyun Ma +2
The potential of large language models (LLMs) in specialized domains such as legal risk analysis remains underexplored. In response to growing interest in locally deploying open-so…