5 papers · 1 filter
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
Yuchen Cai, Ding Cao, Liang Lin +9
On-policy distillation (OPD) has emerged as an efficient post-training paradigm for large language models. However, existing studies largely attribute this advantage to denser and…
On the Superimposed Noise Accumulation Problem in Sequential Knowledge Editing of Large Language Models
Ding Cao, Yuchen Cai, Yuqing Huang +4
Sequential knowledge editing techniques aim to continuously update knowledge in large language models at low cost, preventing models from generating outdated or incorrect informati…
O-Edit: Orthogonal Subspace Editing for Language Model Sequential Editing
Yuchen Cai, Ding Cao
Large language models (LLMs) acquire knowledge during pre-training, but over time, this knowledge may become incorrect or outdated, necessitating updates after training. Knowledge…
Editing Knowledge Representation of Language Model via Rephrased Prefix Prompts
Yuchen Cai, Ding Cao, Rongxi Guo +3
Neural language models (LMs) have been extensively trained on vast corpora to store factual knowledge about various aspects of the world described in texts. Current technologies ty…
Locating and Mitigating Gender Bias in Large Language Models
Yuchen Cai, Ding Cao, Rongxi Guo +3
Large language models(LLM) are pre-trained on extensive corpora to learn facts and human cognition which contain human preferences. However, this process can inadvertently lead to…