On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift

Pratiksha Thaker, Amrith Setlur, Zhiwei Steven Wu +1

Public pretraining is a promising approach to improve differentially private model training. However, recent work has noted that many positive research results studying this paradi…