1 citations · 1 across the 10 of their papers we have counts for
8 papers · 1 filter
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
Sijia Liu, Yicheng Lang, Soumyadeep Pal +6
Zeroth-order (ZO) optimization, learning from finite differences of function evaluations without backpropagation, has recently regained attention in deep learning due to its memory…
Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
Yancheng Huang, Changsheng Wang, Chongyu Fan +7
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, an…
Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
Yicheng Lang, Changsheng Wang, Yihua Zhang +4
Zeroth-order (ZO) optimization provides a gradient-free alternative to first-order (FO) methods by estimating gradients via finite differences of function evaluations, and has rece…
Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization
Changsheng Wang, Xin Chen, Sijia Liu +1
Adapting pretrained large language models (LLMs) to code domains via supervised fine-tuning (FT) has been commonly used for code generation. However, we identify a previously under…
LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data
Changsheng Wang, Yihua Zhang, Dennis Wei +3
Large language models (LLMs) exhibit remarkable generative capabilities but raise ethical and security concerns by memorizing sensitive data, reinforcing biases, and producing harm…
LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics
Chongyu Fan, Changsheng Wang, Yancheng Huang +2
Machine unlearning for large language models (LLMs) aims to remove undesired data, knowledge, and behaviors (e.g., for safety, privacy, or copyright) while preserving useful model…