3 papers
cs.LG2025
Memory-Efficient Differentially Private Training with Gradient Random Projection
Alex Mulrooney, Devansh Gupta, James Flemings +4
Differential privacy (DP) protects sensitive data during neural network training, but standard methods like DP-Adam suffer from high memory overhead due to per-sample gradient clip…
cs.LG2024
Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
Zeman Li, Xinwei Zhang, Peilin Zhong +3
Fine-tuning language models (LMs) with the Adam optimizer often demands excessive memory, limiting accessibility. The "in-place" version of Stochastic Gradient Descent (IP-SGD) and…
cs.LG2024
Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints
Yunsheng Tian, Ane Zuniga, Xinwei Zhang +5
Bayesian optimization has been successfully applied to optimize black-box functions where the number of evaluations is severely limited. However, in many real-world applications, i…