8 papers · 1 filter
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
Qijia He, Ruinan Jin, Jun Luo +2
Asynchronous reinforcement learning (RL) improves the efficiency of large language model post-training but introduces stale rollouts generated by earlier policies. Theoretical unde…
Rethinking AI-Generated Text Detection: A Strong Baseline and the Distribution-Shift Problem That Remains
Zhuoer Shen, Mingyi Wang, Shaofeng Zou +1
Recent AI-generated text detection work often introduces a new benchmark together with a specialized detector tailored to it. We revisit this practice from a baseline-first perspec…
Detector-Evasive LLM Paraphrasing via Constrained Policy Optimization
Mingyi Wang, Zhuoer Shen, Yuheng Bu +1
AI-text detectors are vulnerable to paraphrasing and detector-guided paraphrasing attacks, but existing detector-evasion methods often lack precise control over semantic preservati…
Step-level Denoising-time Diffusion Alignment with Multiple Objectives
Qi Zhang, Dawei Wang, Shaofeng Zou
Reinforcement learning (RL) has emerged as a powerful tool for aligning diffusion models with human preferences, typically by optimizing a single reward function under a KL regular…
HIPO: Instruction Hierarchy via Constrained Reinforcement Learning
Keru Chen, Jun Luo, Sen Lin +4
Hierarchical Instruction Following (HIF) refers to the problem of prompting large language models with a priority-ordered stack of instructions. Standard methods like RLHF and DPO…
LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control
Peiyao Xiao, Chaosheng Dong, Shaofeng Zou +1
Multi-task learning (MTL) has been widely adopted for its ability to simultaneously learn multiple tasks. While existing gradient manipulation methods often yield more balanced sol…
MGDA Converges under Generalized Smoothness, Provably
Qi Zhang, Peiyao Xiao, Shaofeng Zou +1
Multi-objective optimization (MOO) is receiving more attention in various fields such as multi-task learning. Recent works provide some effective algorithms with theoretical analys…
Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning
Yudan Wang, Peiyao Xiao, Hao Ban +2
Multi-task reinforcement learning (MTRL) has shown great promise in many real-world applications. Existing MTRL algorithms often aim to learn a policy that optimizes individual obj…