6 papers
Can Native Safety Travel? Reusing Safety Pretraining Across Models via On-Policy Distillation
Bingqi Shang, Yang Zhang, Yiwei Chen +4
Safety pretraining has emerged as a promising paradigm for embedding safety deeply into a model during pretraining, yielding robustness to subsequent fine-tuning that conventional…
When Retain Constraints Conflict: Mitigating Forget-Retain Interference in Tabular Data
Zijie Liu, Jinhao Duan, Bingqi Shang +3
Machine unlearning aims to remove the influence of designated training data while preserving model utility, but its behavior on tabular data remains underexplored. This gap is impo…
Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
Yiwei Chen, Bingqi Shang, Sijia Liu
Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ow…
Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
Yancheng Huang, Changsheng Wang, Chongyu Fan +7
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, an…
Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
Bingqi Shang, Yiwei Chen, Yihua Zhang +2
Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while retaining their general utility. Yet, with…
Split Adaptation for Pre-trained Vision Transformers
Lixu Wang, Bingqi Shang, Yi Li +4
Vision Transformers (ViTs), extensively pre-trained on large-scale datasets, have become essential to foundation models, allowing excellent performance on diverse downstream tasks…