6 papers
Dual-Channel Robust Group-Relative Policy Optimization via Advantage and Sequence-Weight Estimation
Zhongyi Li, Wan Tian, Xiang Xu +4
Group-relative policy optimization relies on reward-derived advantages and sequence-level likelihood weights, both of which can be sensitive to localized outliers. Extreme rewards…
STAR-GRPO: Canonical Anchoring and Reliability-First Advantages against Representation-Dependent Reward Hacking
Wan Tian, Zhongyi Li, Xiang Xu +3
Reward hacking occurs when policy optimization exploits a brittle reward interface or an overly permissive proxy objective, improving the training score without improving the under…
Beyond Sub-Gaussian Detector Scores: Robust Weighted Profile-Loss Change Point Detection for Human-LLM Text Segmentation
Wan Tian, Zhongyi Li, Yawen Li +3
Mixed human-LLM documents require locating authorship transitions from detector scores whose reliability varies across text units. Existing weighted mean contrasts are vulnerable t…
Adaptive Robust Estimator for Multi-Agent Reinforcement Learning
Zhongyi Li, Wan Tian, Jingyu Chen +8
Multi-agent collaboration has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models, yet it suffers from interaction-level ambiguity that…
Sharper Generalization Bounds for Transformer
Yawen Li, Tao Hu, Zhouhui Lian +4
This paper studies generalization error bounds for Transformer models. Based on the offset Rademacher complexity, we derive sharper generalization bounds for different Transformer…
Counterfactual Credit Policy Optimization for Multi-Agent Collaboration
Zhongyi Li, Wan Tian, Jinju Chen +4
Collaborative multi-agent large language models (LLMs) can solve complex reasoning tasks by decomposing roles, but reinforcement learning for such systems is limited by credit assi…