22 citations · 102 across the 31 of their papers we have counts for
21 papers · 1 filter
Towards hyperparameter-free optimization with differential privacy
Zhiqi Bu, Ruixuan Liu
Differential privacy (DP) is a privacy-preserving paradigm that protects the training data when training deep learning models. Critically, the performance of models is determined b…
DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction
Xinwei Zhang, Zhiqi Bu, Mingyi Hong +1
Privacy is a growing concern in modern deep-learning systems and applications. Differentially private (DP) training prevents the leakage of sensitive information in the collected t…
Pre-training Differentially Private Models with Limited Public Data
Zhiqi Bu, Xinwei Zhang, Mingyi Hong +2
The superior performance of large foundation models relies on the use of massive amounts of high-quality data, which often contain sensitive, private and copyrighted material that…
Zero redundancy distributed learning with differential privacy
Zhiqi Bu, Justin Chiu, Ruixuan Liu +2
Deep learning using large models have achieved great success in a wide range of domains. However, training these models on billions of parameters is very challenging in terms of th…
Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach
Xinwei Zhang, Zhiqi Bu, Zhiwei Steven Wu +1
Differentially Private Stochastic Gradient Descent with Gradient Clipping (DPSGD-GC) is a powerful tool for training deep learning models using sensitive data, providing both a sol…
On the accuracy and efficiency of group-wise clipping in differentially private optimization
Zhiqi Bu, Ruixuan Liu, Yu-Xiang Wang +2
Recent advances have substantially improved the accuracy, memory cost, and training speed of differentially private (DP) deep learning, especially on large vision and language mode…
Coupling public and private gradient provably helps optimization
Ruixuan Liu, Zhiqi Bu, Yu-xiang Wang +2
The success of large neural networks is crucially determined by the availability of data. It has been observed that training only on a small amount of public data, or privately on…
Multiple Imputation with Neural Network Gaussian Process for High-dimensional Incomplete Data
Zongyu Dai, Zhiqi Bu, Qi Long
Missing data are ubiquitous in real world applications and, if not adequately handled, may lead to the loss of information and biased findings in downstream analysis. Particularly,…
Accelerating Adversarial Perturbation by 50% with Semi-backward Propagation
Zhiqi Bu
Adversarial perturbation plays a significant role in the field of adversarial robustness, which solves a maximization problem over the input data. We show that the backward propaga…
Differentially Private Optimizers Can Learn Adversarially Robust Models
Yuan Zhang, Zhiqi Bu
Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities. One important yet worrying questio…
Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
Differentially private (DP) optimization is the standard paradigm to learn large neural networks that are accurate and privacy-preserving. The computational cost for DP deep learni…
Differentially Private Bias-Term Fine-tuning of Foundation Models
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
We study the problem of differentially private (DP) fine-tuning of large pre-trained models -- a recent privacy-preserving approach suitable for solving downstream tasks with sensi…
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
Per-example gradient clipping is a key algorithmic step that enables practical differential private (DP) training for deep learning models. The choice of clipping threshold R, howe…
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Zhiqi Bu, Jialin Mao, Shiyun Xu
Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive…
Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problems
Zongyu Dai, Zhiqi Bu, Qi Long
Missing data are present in most real world problems and need careful handling to preserve the prediction accuracy and statistical consistency in the downstream analysis. As the go…
Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability
Qiyiwen Zhang, Zhiqi Bu, Kan Chen +1
Bayesian neural network (BNN) allows for uncertainty quantification in prediction, offering an advantage over regular neural networks that has not been explored in the differential…
Accuracy, Interpretability, and Differential Privacy via Explainable Boosting
Harsha Nori, Rich Caruana, Zhiqi Bu +2
We show that adding differential privacy to Explainable Boosting Machines (EBMs), a recent method for training interpretable ML models, yields state-of-the-art accuracy while prote…
On the Convergence and Calibration of Deep Learning with Differential Privacy
Zhiqi Bu, Hua Wang, Zongyu Dai +1
Differentially private (DP) training preserves the data privacy usually at the cost of slower convergence (and thus lower accuracy), as well as more severe mis-calibration than its…
Fast and Memory Efficient Differentially Private-SGD via JL Projections
Zhiqi Bu, Sivakanth Gopi, Janardhan Kulkarni +3
Differentially Private-SGD (DP-SGD) of Abadi et al. (2016) and its variations are the only known algorithms for private training of large scale neural networks. This algorithm requ…
A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks
Zhiqi Bu, Shiyun Xu, Kan Chen
When equipped with efficient optimization algorithms, the over-parameterized neural networks have demonstrated high level of performance even though the loss function is non-convex…
Deep Learning with Gaussian Differential Privacy
Zhiqi Bu, Jinshuo Dong, Qi Long +1
Deep learning models are often trained on datasets that contain sensitive information such as individuals' shopping transactions, personal contacts, and medical records. An increas…