47 citations · 214 across the 48 of their papers we have counts for
9 papers · 1 filter
Optimizing Information-theoretical Generalization Bounds via Anisotropic Noise in SGLD
Bohan Wang, Huishuai Zhang, Jieyu Zhang +3
Recently, the information-theoretical framework has been proven to be able to obtain non-vacuous generalization bounds for large models trained by Stochastic Gradient Langevin Dyna…
Availability Attacks Create Shortcuts
Da Yu, Huishuai Zhang, Wei Chen +2
Availability attacks, which poison the training data with imperceptible perturbations, can make the data \emph{not exploitable} by machine learning algorithms so as to prevent unau…
Differentially Private Fine-tuning of Language Models
Da Yu, Saurabh Naik, Arturs Backurs +9
We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus…
Does Momentum Change the Implicit Regularization on Separable Data?
Bohan Wang, Qi Meng, Huishuai Zhang +4
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…
Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit
Yichi Zhou, Shihong Song, Huishuai Zhang +3
Balancing exploration and exploitation (EE) is a fundamental problem in contex-tual bandit. One powerful principle for EE trade-off isOptimism in Face of Uncer-tainty(OFU), in whic…
Large Scale Private Learning via Low-rank Reparametrization
Da Yu, Huishuai Zhang, Wei Chen +2
We propose a reparametrization scheme to address the challenges of applying differentially private SGD on large neural networks, which are 1) the huge memory cost of storing indivi…