8 papers · 1 filter
Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
Leitian Tao, Ilia Kulikov, Swarnadeep Saha +5
Post-training for reasoning of large language models (LLMs) increasingly relies on verifiable rewards: deterministic checkers that provide 0-1 correctness signals. While reliable,…
J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning
Chenxi Whitehouse, Tianlu Wang, Ping Yu +4
The progress of AI is bottlenecked by the quality of evaluation, making powerful LLM-as-a-Judge models a core solution. The efficacy of these judges depends on their chain-of-thoug…
Jointly Reinforcing Diversity and Quality in Language Model Generations
Tianjian Li, Yiming Zhang, Ping Yu +5
Post-training of Large Language Models (LMs) often prioritizes accuracy and helpfulness at the expense of diversity. This creates a tension: while post-training improves response q…
Multi-Token Attention
Olga Golovneva, Tianlu Wang, Jason Weston +1
Soft attention is a critical mechanism powering LLMs to locate relevant parts within a given context. However, individual attention weights are determined by the similarity of only…
Bridging Offline and Online Reinforcement Learning for LLMs
Jack Lanchantin, Angelica Chen, Janice Lan +9
We investigate the effectiveness of reinforcement learning methods for finetuning large language models when transitioning from offline to semi-online to fully online regimes for b…
Efficient Tool Use with Chain-of-Abstraction Reasoning
Silin Gao, Jane Dwivedi-Yu, Ping Yu +7
To achieve faithful reasoning that aligns with human expectations, large language models (LLMs) need to ground their reasoning to real-world knowledge (e.g., web facts, math and ph…