47 citations · 145 across the 24 of their papers we have counts for
8 papers · 1 filter
Privately Aligning Language Models with Reinforcement Learning
Fan Wu, Huseyin A. Inan, Arturs Backurs +3
Positioned between pre-training and user deployment, aligning large language models (LLMs) through reinforcement learning (RL) has emerged as a prevailing strategy for training ins…
Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation
Xinyu Tang, Richard Shin, Huseyin A. Inan +6
We study the problem of in-context learning (ICL) with large language models (LLMs) on private datasets. This scenario poses privacy risks, as LLMs may leak or regurgitate the priv…
When Does Differentially Private Learning Not Suffer in High Dimensions?
Xuechen Li, Daogao Liu, Tatsunori Hashimoto +4
Large pretrained models can be privately fine-tuned to achieve performance approaching that of non-private models. A common theme in these results is the surprising observation tha…
Differentially Private Model Compression
Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan +2
Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models…
Differentially Private Fine-tuning of Language Models
Da Yu, Saurabh Naik, Arturs Backurs +9
We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus…
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa +3
Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy implications when training models on user content such…
rTop-k: A Statistical Estimation Approach to Distributed SGD
Leighton Pate Barnes, Huseyin A. Inan, Berivan Isik +1
The large communication cost for exchanging gradients between different nodes significantly limits the scalability of distributed training for large-scale learning models. Motivate…
Adaptive Mixture Methods Based on Bregman Divergences
Mehmet A. Donmez, Huseyin A. Inan, Suleyman S. Kozat
We investigate adaptive mixture methods that linearly combine outputs of constituent filters running in parallel to model a desired signal. We use "Bregman divergences" and obt…