11 papers
Generalizable Model Editing Could Be More Than Direct Generalization
Xiaotian Ye, Chunyao Yang, Xiaohan Wang +4
A central desideratum in model editing is generalization to complex tasks, which prior work formalizes under a direct generalization paradigm: treating and evaluating LLM behavior…
On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning
Xiaotian Ye, Xiaohan Wang, Mengqi Zhang +1
Counterfactual tuning (CFT) has emerged as a promising paradigm for Large Language Model (LLM) unlearning by training models to generate alternative fictitious knowledge in place o…
Uncovering Entity Identity Confusion in Multimodal Knowledge Editing
Shu Wu, Xiaotian Ye, Xinyu Mou +3
Multimodal knowledge editing (MKE) aims to correct the internal knowledge of large vision-language models after deployment, yet the behavioral patterns of post-edit models remain u…
Uncovering Context Reliance in Unstructured Knowledge Editing
Zisheng Zhou, Mengqi Zhang, Shiguang Wu +4
Editing Large language models (LLMs) with real-world, unstructured knowledge is essential for correcting and updating their internal parametric knowledge. In this work, we revisit…
Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse
Chi Zhang, Mengqi Zhang, Xiaotian Ye +5
Sequential knowledge editing in large language models often causes catastrophic collapse of the model's general abilities, especially for parameter-modifying methods. Existing appr…
LLM Unlearning Should Be Form-Independent
Xiaotian Ye, Mengqi Zhang, Shu Wu
Large Language Model (LLM) unlearning aims to erase or suppress undesirable knowledge within the model, offering promise for controlling harmful or private information to prevent m…
Disentangling Knowledge Representations for Large Language Model Editing
Mengqi Zhang, Zisheng Zhou, Xiaotian Ye +4
Knowledge Editing has emerged as a promising solution for efficiently updating embedded knowledge in large language models (LLMs). While existing approaches demonstrate effectivene…
Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms
Xiaotian Ye, Mengqi Zhang, Shu Wu
Knowledge is fundamental to the overall capabilities of Large Language Models (LLMs). The knowledge paradigm of a model, which dictates how it encodes and utilizes knowledge, signi…
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets
Wenyu Wang, Mengqi Zhang, Xiaotian Ye +3
Large Language Models (LLMs) inevitably acquire harmful information during training on massive datasets. LLM unlearning aims to eliminate the influence of such harmful information…
Uncovering Overfitting in Large Language Model Editing
Mengqi Zhang, Xiaotian Ye, Qiang Liu +3
Knowledge editing has been proposed as an effective method for updating and correcting the internal knowledge of Large Language Models (LLMs). However, existing editing methods oft…
Knowledge Graph Enhanced Large Language Model Editing
Mengqi Zhang, Xiaotian Ye, Qiang Liu +3
Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and outdated knowledge. Model editing…