4 papers · 1 filter
Can Native Safety Travel? Reusing Safety Pretraining Across Models via On-Policy Distillation
Bingqi Shang, Yang Zhang, Yiwei Chen +4
Safety pretraining has emerged as a promising paradigm for embedding safety deeply into a model during pretraining, yielding robustness to subsequent fine-tuning that conventional…
When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Data
Zijie Liu, Jinhao Duan, Bingqi Shang +3
Machine unlearning aims to remove the influence of designated training data while preserving model utility, but its behavior on tabular data remains underexplored. This gap is impo…
Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
Yancheng Huang, Changsheng Wang, Chongyu Fan +7
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, an…
Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
Bingqi Shang, Yiwei Chen, Yihua Zhang +2
Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while retaining their general utility. Yet, with…