8 papers
Encoding Context into LLM Weights: What Makes It Context?
Yue Wang, Qizhou Wang, Feng Liu +3
Longer contexts allow large language models (LLMs) to condition their predictions on more information, but inference costs grow prohibitively with context length. An emerging alter…
Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding
Zhiqin Yang, Yuhan Liu, Jingwen Fu +4
Although natural language is the default medium for Large Language Models (LLMs), its limited expressive capacity creates a profound bottleneck for complex problem-solving. While r…
Understanding Negative Learning in Large Language Model Fine-tuning
Yue Wang, Qizhou Wang, Zizhuo Zhang +3
Negative learning has never been more important than it is today, particularly in large language model (LLM) fine-tuning, where it underpins machine unlearning, preference- and rew…
Learning without Isolation: Pathway Protection for Continual Learning
Zhikang Chen, Abudukelimu Wuerkaixi, Sen Cui +10
Deep networks are prone to catastrophic forgetting during sequential task learning, i.e., losing the knowledge about old tasks upon learning new tasks. To this end, continual learn…
Accurate Forgetting for Heterogeneous Federated Continual Learning
Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang +6
Recent years have witnessed a burgeoning interest in federated learning (FL). However, the contexts in which clients engage in sequential learning remain under-explored. Bridging F…
Decoupling the Class Label and the Target Concept in Machine Unlearning
Jianing Zhu, Bo Han, Jiangchao Yao +3
Machine unlearning as an emerging research topic for data regulations, aims to adjust a trained model to approximate a retrained one that excludes a portion of training data. Previ…
Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
Qizhou Wang, Bo Han, Puning Yang +3
The imperative to eliminate undesirable data memorization underscores the significance of machine unlearning for large language models (LLMs). Recent research has introduced a seri…
Balancing Similarity and Complementarity for Federated Learning
Kunda Yan, Sen Cui, Abudukelimu Wuerkaixi +5
In mobile and IoT systems, Federated Learning (FL) is increasingly important for effectively using data while maintaining user privacy. One key challenge in FL is managing statisti…