5 papers · 1 filter
BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents
Hanyang Wang, Weijieying Ren, Yuxiang Zhang +4
Stepwise group-based RL is an attractive way to train long-horizon LLM agents without a learned critic: it reuses multiple sampled rollouts to estimate local advantages. Its weakne…
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
Yuchen Cai, Ding Cao, Liang Lin +9
On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, existing studies largely attribute this advantage to denser and…
On the Superimposed Noise Accumulation Problem in Sequential Knowledge Editing of Large Language Models
Ding Cao, Yuchen Cai, Yuqing Huang +4
Sequential knowledge editing techniques aim to continuously update knowledge in large language models at low cost, preventing models from generating outdated or incorrect informati…
O-Edit: Orthogonal Subspace Editing for Language Model Sequential Editing
Yuchen Cai, Ding Cao
Large language models (LLMs) acquire knowledge during pre-training, but over time, this knowledge may become incorrect or outdated, necessitating updates after training. Knowledge…
Editing Knowledge Representation of Language Model via Rephrased Prefix Prompts
Yuchen Cai, Ding Cao, Rongxi Guo +3
Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies ty…