4 papers
Characterizing the Training Dynamics of Private Fine-tuning with Langevin diffusion
Shuqi Ke, Charlie Hou, Sewoong Oh +1
We show that differentially private full fine-tuning (DP-FFT) can distort pre-trained backbone features based on both theoretical and empirical results. We identify the cause of th…
Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity
Charlie Hou, Kiran Koshy Thekumparampil, Michael Shavlovsky +3
On tabular data, a significant body of literature has shown that current deep learning (DL) models perform at best similarly to Gradient Boosted Decision Trees (GBDTs), while signi…
POPri: Private Federated Learning using Preference-Optimized Synthetic Data
Charlie Hou, Mei-Yu Wang, Yige Zhu +2
In practical settings, differentially private Federated learning (DP-FL) is the dominant method for training models from private, on-device client data. Recent work has suggested t…
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs
Charlie Hou, Akshat Shrivastava, Hongyuan Zhan +5
On-device training is currently the most common approach for training machine learning (ML) models on private, distributed user data. Despite this, on-device training has several d…