Can Public Large Language Models Help Private Cross-device Federated Learning?

Boxin Wang, Yibo Jacky Zhang, Yuan Cao +5

We study (differentially) private federated learning (FL) of language models. The language models in cross-device FL are relatively small, which can be trained with meaningful form…