1 citations · 1 across the 7 of their papers we have counts for
10 papers
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
Yuxiang Zhang, Ding Cao, Shuting Cui +3
On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-w…
Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors
Yuchen Cai, Ding Cao, Qixiang Yin +10
Reinforcement learning (RL) has become a key paradigm for enhancing the reasoning of large language models, yet the high dimensionality of parameter updates makes its training dyna…
SAGE: Structured Strategic Reasoning for Efficient LLM Game Playing
Zhiwei Chen, Tianchun Wang, Zhongtao Rao +3
A strong LLM strategic agent should reason prospectively over uncertain futures, adapt its strategy to opponents' behavioral tendencies, and continuously recalibrate its decision p…
BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents
Hanyang Wang, Weijieying Ren, Yuxiang Zhang +4
Stepwise group-based RL is an attractive way to train long-horizon LLM agents without a learned critic: it reuses multiple sampled rollouts to estimate local advantages. Its weakne…
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
Yuchen Cai, Ding Cao, Liang Lin +9
On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, existing studies largely attribute this advantage to denser and…
On Predictability of Reinforcement Learning Dynamics for Large Language Models
Yuchen Cai, Ding Cao, Xin Xu +7
Recent advances in reasoning capabilities of large language models (LLMs) are largely driven by reinforcement learning (RL), yet the underlying parameter dynamics during RL trainin…
On the Superimposed Noise Accumulation Problem in Sequential Knowledge Editing of Large Language Models
Ding Cao, Yuchen Cai, Yuqing Huang +4
Sequential knowledge editing techniques aim to continuously update knowledge in large language models at low cost, preventing models from generating outdated or incorrect informati…
O-Edit: Orthogonal Subspace Editing for Language Model Sequential Editing
Yuchen Cai, Ding Cao
Large language models (LLMs) acquire knowledge during pre-training, but over time, this knowledge may become incorrect or outdated, necessitating updates after training. Knowledge…
Locating and Mitigating Gender Bias in Large Language Models
Yuchen Cai, Ding Cao, Rongxi Guo +3
Large language models(LLM) are pre-trained on extensive corpora to learn facts and human cognition which contain human preferences. However, this process can inadvertently lead to…
Editing Knowledge Representation of Language Model via Rephrased Prefix Prompts
Yuchen Cai, Ding Cao, Rongxi Guo +3
Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies ty…