Practical Secure Aggregation for Federated Learning on User-Held Data

Abstract

Secure Aggregation protocols allow a collection of mutually distrust parties, each holding a private value, to collaboratively compute the sum of those values without revealing the values themselves. We consider training a deep neural network in the Federated Learning model, using distributed stochastic gradient descent across user-held training data on mobile devices, wherein Secure Aggregation protects each user's model gradient. We design a novel, communication-efficient Secure Aggregation protocol for high-dimensional data that tolerates up to 1/3 users failing to complete the protocol. For 16-bit input values, our protocol offers 1.73x communication expansion for $2^{10}$ users and $2^{20}$-dimensional vectors, and 1.98x expansion for $2^{14}$ users and $2^{24}$ dimensional vectors.

5 pages, 1 figure. To appear at the NIPS 2016 workshop on Private Multi-Party Machine Learning

References in corpus (2)

• Deep Learning with Differential Privacy
• Proactively Accountable Anonymous Messaging in Verdict

Cited by in corpus (64)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
• Federated Learning for Mobile Keyboard Prediction
• Privacy-preserving Traffic Flow Prediction: A Federated Learning Approach
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Analysis and Optimal Edge Assignment For Hierarchical Federated Learning on Non-IID Data
• Distributed Mean Estimation with Limited Communication
• Overcoming Forgetting in Federated Learning on Non-IID Data
• GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning
• Abnormal Client Behavior Detection in Federated Learning
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Federated Learning Of Out-Of-Vocabulary Words
• Differentially Private Learning with Adaptive Clipping
• Adaptive Federated Learning in Resource Constrained Edge Computing Systems
• Distilled One-Shot Federated Learning
• Free-riders in Federated Learning: Attacks and Defenses
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• Federated Learning with Only Positive Labels
• Federated Learning for Ranking Browser History Suggestions
• Federated Learning with Buffered Asynchronous Aggregation
• Stochastic, Distributed and Federated Optimization for Machine Learning
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Papaya: Practical, Private, and Scalable Federated Learning
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
• Deep Leakage from Gradients
• Federated Dynamic GNN with Secure Aggregation
• Towards Federated Graph Learning for Collaborative Financial Crimes Detection
• Federated Bandit: A Gossiping Approach
• InstaHide: Instance-hiding Schemes for Private Distributed Learning
• SPEED: Secure, PrivatE, and Efficient Deep learning
• Privacy attacks for automatic speech recognition acoustic models in a federated learning framework
• Federated Learning for Smart Healthcare: A Survey
• A Theoretical Perspective on Differentially Private Federated Multi-task Learning
• Incentives for Federated Learning: a Hypothesis Elicitation Approach
• Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• LoAdaBoost: loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data
• Secure Aggregation for Buffered Asynchronous Federated Learning
• Federated Heavy Hitters Discovery with Differential Privacy
• FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• Accelerating Federated Learning via Momentum Gradient Descent
• Is Private Learning Possible with Instance Encoding?
• SoK: Machine Learning Governance
• FedSiam: Towards Adaptive Federated Semi-Supervised Learning
• Federated Learning in Mobile Edge Computing: An Edge-Learning Perspective for Beyond 5G
• Concentrated Differentially Private and Utility Preserving Federated Learning
• Distributed Differentially Private Computation of Functions with Correlated Noise
• Confederated Machine Learning on Horizontally and Vertically Separated Medical Data for Large-Scale Health System Intelligence
• Efficient Privacy Preserving Edge Computing Framework for Image Classification
• OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning
• Federated Learning in Adversarial Settings
• Federated Learning for Coalition Operations
• The Need for Speed of AI Applications: Performance Comparison of Native vs. Browser-based Algorithm Implementations
• Trends and Advancements in Deep Neural Network Communication
• Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
• Cost-Effective Federated Learning Design
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices
• Glimmers: Resolving the Privacy/Trust Quagmire
• An Enhanced Approach to Cloud-based Privacy-preserving Benchmarking (Long Version)
• A Comparison of Methods for Neural Network Aggregation