22 citations · 102 across the 31 of their papers we have counts for
31 papers
Sharp Trade-Offs in High-Dimensional Inference via 2-Level SLOPE
Zhiqi Bu, Jason M. Klusowski, Cynthia Rush +1
Among techniques for high-dimensional linear regression, Sorted L-One Penalized Estimation (SLOPE) generalizes the LASSO via an adaptive regularization that applies heavier p…
Towards hyperparameter-free optimization with differential privacy
Zhiqi Bu, Ruixuan Liu
Differential privacy (DP) is a privacy-preserving paradigm that protects the training data when training deep learning models. Critically, the performance of models is determined b…
DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction
Xinwei Zhang, Zhiqi Bu, Mingyi Hong +1
Privacy is a growing concern in modern deep-learning systems and applications. Differentially private (DP) training prevents the leakage of sensitive information in the collected t…
Pre-training Differentially Private Models with Limited Public Data
Zhiqi Bu, Xinwei Zhang, Mingyi Hong +2
The superior performance of large foundation models relies on the use of massive amounts of high-quality data, which often contain sensitive, private and copyrighted material that…
Zero redundancy distributed learning with differential privacy
Zhiqi Bu, Justin Chiu, Ruixuan Liu +2
Deep learning using large models have achieved great success in a wide range of domains. However, training these models on billions of parameters is very challenging in terms of th…
Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach
Xinwei Zhang, Zhiqi Bu, Zhiwei Steven Wu +1
Differentially Private Stochastic Gradient Descent with Gradient Clipping (DPSGD-GC) is a powerful tool for training deep learning models using sensitive data, providing both a sol…
On the accuracy and efficiency of group-wise clipping in differentially private optimization
Zhiqi Bu, Ruixuan Liu, Yu-Xiang Wang +2
Recent advances have substantially improved the accuracy, memory cost, and training speed of differentially private (DP) deep learning, especially on large vision and language mode…
Coupling public and private gradient provably helps optimization
Ruixuan Liu, Zhiqi Bu, Yu-xiang Wang +2
The success of large neural networks is crucially determined by the availability of data. It has been observed that training only on a small amount of public data, or privately on…
MISNN: Multiple Imputation via Semi-parametric Neural Networks
Zhiqi Bu, Zongyu Dai, Yiliang Zhang +1
Multiple imputation (MI) has been widely applied to missing value problems in biomedical, social and econometric research, in order to avoid improper inference in the downstream da…
Multiple Imputation with Neural Network Gaussian Process for High-dimensional Incomplete Data
Zongyu Dai, Zhiqi Bu, Qi Long
Missing data are ubiquitous in real world applications and, if not adequately handled, may lead to the loss of information and biased findings in downstream analysis. Particularly,…
Accelerating Adversarial Perturbation by 50% with Semi-backward Propagation
Zhiqi Bu
Adversarial perturbation plays a significant role in the field of adversarial robustness, which solves a maximization problem over the input data. We show that the backward propaga…
Differentially Private Optimizers Can Learn Adversarially Robust Models
Yuan Zhang, Zhiqi Bu
Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities. One important yet worrying questio…
Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
Differentially private (DP) optimization is the standard paradigm to learn large neural networks that are accurate and privacy-preserving. The computational cost for DP deep learni…
Differentially Private Bias-Term Fine-tuning of Foundation Models
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
We study the problem of differentially private (DP) fine-tuning of large pre-trained models -- a recent privacy-preserving approach suitable for solving downstream tasks with sensi…
CEDAR: Communication Efficient Distributed Analysis for Regressions
Changgee Chang, Zhiqi Bu, Qi Long
Electronic health records (EHRs) offer great promises for advancing precision medicine and, at the same time, present significant analytical challenges. Particularly, it is often t…
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha +1
Per-example gradient clipping is a key algorithmic step that enables practical differential private (DP) training for deep learning models. The choice of clipping threshold R, howe…
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Zhiqi Bu, Jialin Mao, Shiyun Xu
Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive…
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity
Shiyun Xu, Zhiqi Bu, Pratik Chaudhari +1
Interpretable machine learning has demonstrated impressive performance while preserving explainability. In particular, neural additive models (NAM) offer the interpretability to th…
Multiple Imputation via Generative Adversarial Network for High-dimensional Blockwise Missing Value Problems
Zongyu Dai, Zhiqi Bu, Qi Long
Missing data are present in most real world problems and need careful handling to preserve the prediction accuracy and statistical consistency in the downstream analysis. As the go…
Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability
Qiyiwen Zhang, Zhiqi Bu, Kan Chen +1
Bayesian neural network (BNN) allows for uncertainty quantification in prediction, offering an advantage over regular neural networks that has not been explored in the differential…
Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing Algorithm
Kan Chen, Zhiqi Bu, Shiyun Xu
Sparse Group LASSO (SGL) is a regularized model for high-dimensional linear regression problems with grouped covariates. SGL applies and penalties on the individual pre…
Privacy Amplification via Iteration for Shuffled and Online PNSGD
Matteo Sordello, Zhiqi Bu, Jinshuo Dong
In this paper, we consider the framework of privacy amplification via iteration, which is originally proposed by Feldman et al. and subsequently simplified by Asoodeh et al. in the…
Accuracy, Interpretability, and Differential Privacy via Explainable Boosting
Harsha Nori, Rich Caruana, Zhiqi Bu +2
We show that adding differential privacy to Explainable Boosting Machines (EBMs), a recent method for training interpretable ML models, yields state-of-the-art accuracy while prote…
On the Convergence and Calibration of Deep Learning with Differential Privacy
Zhiqi Bu, Hua Wang, Zongyu Dai +1
Differentially private (DP) training preserves the data privacy usually at the cost of slower convergence (and thus lower accuracy), as well as more severe mis-calibration than its…