6 papers
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense
Yushu Li, Jiin Woo, Xiaoxiao Li +1
To improve task performance while encouraging desirable response properties, LLM post-training often combines a primary reward (e.g., correctness) with auxiliary rewards (e.g., bre…
Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
Wenlong Deng, Jiaji Huang, Kaan Ozkara +4
Reward hacking arises when a model improves a proxy reward by exploiting shortcuts rather than solving the intended task. We study this failure mode through the geometry of reinfor…
Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents
Yushu Li, Wenlong Deng, Jiajin Li +1
Test-time scaling has become a dominant paradigm for improving LLM agent reliability, yet current approaches treat compute as an abundant resource, allowing agents to exhaust token…
When RAG Hurts: Diagnosing and Mitigating Attention Distraction in Retrieval-Augmented LVLMs
Beidi Zhao, Wenlong Deng, Xinting Liao +4
While Retrieval-Augmented Generation (RAG) is one of the dominant paradigms for enhancing Large Vision-Language Models (LVLMs) on knowledge-based VQA tasks, recent work attributes…
On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacement
Wenlong Deng, Yushu Li, Boying Gong +3
Tool-integrated (TI) reinforcement learning (RL) enables large language models (LLMs) to perform multi-step reasoning by interacting with external tools such as search engines and…
Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learning
Wenlong Deng, Yi Ren, Yushu Li +4
Reinforcement learning with verifiable rewards has significantly advanced the reasoning capabilities of large language models, yet how to explicitly steer training toward explorati…