5 papers
DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off
Xiaofan Li, Ming Yang, Zhiyuan Ma +9
Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed significant advances in the reasoning capabilities of Large Language Models (LLMs). However, effectively managin…
TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimization
Shichao Ma, Zhiyuan Ma, Ming Yang +8
Multi-turn tool-integrated reasoning enables Large Language Models (LLMs) to solve complex tasks through iterative information retrieval. However, current reinforcement learning (R…
HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement
Ming Yang, Xiaofan Li, Zhiyuan Ma +4
Recent curriculum reinforcement learning for large language models (LLMs) typically rely on difficulty-based annotations for data filtering and ordering. However, such methods suff…
Mirror-Consistency: Harnessing Inconsistency in Majority Voting
Siyuan Huang, Zhiyuan Ma, Jintao Du +3
Self-Consistency, a widely-used decoding strategy, significantly boosts the reasoning capabilities of Large Language Models (LLMs). However, it depends on the plurality voting rule…
Gumbel Reranking: Differentiable End-to-End Reranker Optimization
Siyuan Huang, Zhiyuan Ma, Jintao Du +5
RAG systems rely on rerankers to identify relevant documents. However, fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs. Existing…