12 citations · 30 across the 19 of their papers we have counts for
4 papers · 1 filter
Crafting Reversible SFT Behaviors in Large Language Models
Yuping Lin, Pengfei He, Yue Xing +5
Supervised fine-tuning (SFT) induces new behaviors in large language models, yet imposes no structural constraint on how these behaviors are distributed within the model. Existing…
SoK: Machine Unlearning for Large Language Models
Jie Ren, Yue Xing, Yingqian Cui +2
Large language model (LLM) unlearning has become a critical topic in machine learning, aiming to eliminate the influence of specific training data or knowledge without retraining t…
Towards the Effect of Examples on In-Context Learning: A Theoretical Case Study
Pengfei He, Yingqian Cui, Han Xu +4
In-context learning (ICL) has emerged as a powerful capability for large language models (LLMs) to adapt to downstream tasks by leveraging a few (demonstration) examples. Despite i…
Superiority of Multi-Head Attention in In-Context Linear Regression
Yingqian Cui, Jie Ren, Pengfei He +2
We present a theoretical analysis of the performance of transformer with softmax attention in in-context learning with linear regression tasks. While the existing literature predom…