4 papers · 1 filter
Scaling depth capacity via zero/one-layer model expansion
Zhiqi Bu
Model depth is a double-edged sword in deep learning: deeper models achieve higher accuracy but require higher computational cost. To efficiently train models at scale, progressive…
BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu +6
Enabling large language models (LLMs) to unlearn knowledge and capabilities acquired during training has proven vital for ensuring compliance with data regulations and promoting et…
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri +4
Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspectiv…
DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction
Xinwei Zhang, Zhiqi Bu, Borja Balle +3
Differential privacy (DP) offers a robust framework for safeguarding individual data privacy. To utilize DP in training modern machine learning models, differentially private optim…