6 papers
Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space
Yiwei Chen, Bingqi Shang, Sijia Liu
Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ow…
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
Sijia Liu, Yicheng Lang, Soumyadeep Pal +6
Zeroth-order (ZO) optimization, learning from finite differences of function evaluations without backpropagation, has recently regained attention in deep learning due to its memory…
Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
Bingqi Shang, Yiwei Chen, Yihua Zhang +2
Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while retaining their general utility. Yet, with…
One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
Mohan Zhang, Yihua Zhang, Jinghan Jia +3
Modern large reasoning models (LRMs) exhibit impressive multi-step problem-solving via chain-of-thought (CoT) reasoning. However, this iterative thinking mechanism introduces a new…
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
Hongkang Li, Yihua Zhang, Shuai Zhang +3
Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors, each of which is the weight update from the pre-trained model to fine-tuned models…
Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing
Hanhui Wang, Yihua Zhang, Ruizheng Bai +3
Recent advancements in diffusion models have made generative image editing more accessible, enabling creative edits but raising ethical concerns, particularly regarding malicious e…