Differentially Private Model Compression

Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan +2

Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models…