Enhancing the Privacy of Federated Learning with Sketching

Abstract

In response to growing concerns about user privacy, federated learning has emerged as a promising tool to train statistical models over networks of devices while keeping data localized. Federated learning methods run training tasks directly on user devices and do not share the raw user data with third parties. However, current methods still share model updates, which may contain private information (e.g., one's weight and height), during the training process. Existing efforts that aim to improve the privacy of federated learning make compromises in one or more of the following key areas: performance (particularly communication cost), accuracy, or privacy. To better optimize these trade-offs, we propose that \textit{sketching algorithms} have a unique advantage in that they can provide both privacy and performance benefits while maintaining accuracy. We evaluate the feasibility of sketching-based federated learning with a prototype on three representative learning models. Our initial findings show that it is possible to provide strong privacy guarantees for federated learning without sacrificing performance or accuracy. Our work highlights that there exists a fundamental connection between privacy and communication in distributed settings, and suggests important open problems surrounding the theoretical understanding, methodology, and system design of practical, private federated learning.

References in corpus (11)

• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning for Mobile Keyboard Prediction
• Learning Differentially Private Recurrent Language Models
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Differentially Private Federated Learning: A Client Level Perspective
• EMNIST: an extension of MNIST to handwritten letters
• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• LEAF: A Benchmark for Federated Settings
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketches

Cited by in corpus (5)

• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices
• Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketches
• FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
• FLFE: A Communication-Efficient and Privacy-Preserving Federated Feature Engineering Framework