Training Production Language Models without Memorizing User Data

Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3

This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…