5 papers
How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
Minghao Tian, Yunfei Xie, Chen Wei
Group Relative Policy Optimization (GRPO) has been a key driver of recent progress in reinforcement learning with verifiable rewards (RLVR) for large language models, but it is typ…
InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
Chengwei Wei, Jung-jae Kim, Longyin Zhang +2
Large Language Models (LLMs) with extended reasoning capabilities often generate verbose and redundant reasoning traces, incurring unnecessary computational cost. While existing re…
Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning
Zhiling Ye, Yun Yue, Haowen Wang +11
Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and gener…
Learning to Align, Aligning to Learn: A Unified Approach for Self-Optimized Alignment
Haowen Wang, Yun Yue, Zhiling Ye +12
Alignment methodologies have emerged as a critical pathway for enhancing language model alignment capabilities. While SFT (supervised fine-tuning) accelerates convergence through d…
Confidence-Aware Sub-Structure Beam Search (CABS): Mitigating Hallucination in Structured Data Generation with Large Language Models
Chengwei Wei, Kee Kiat Koo, Amir Tavanaei +1
Large Language Models (LLMs) have facilitated structured data generation, with applications in domains like tabular data, document databases, product catalogs, etc. However, concer…