Differentially Private Federated Learning: A Client Level Perspective

Abstract

Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance.

NIPS 2017 Workshop: Machine Learning on the Phone and other Consumer Devices

Cited by in corpus (196)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• On the Convergence of FedAvg on Non-IID Data
• How To Backdoor Federated Learning
• Federated Machine Learning: Concept and Applications
• Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
• Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy
• From Distributed Machine Learning to Federated Learning: A Survey
• Mitigating Sybils in Federated Learning Poisoning
• FedML: A Research Library and Benchmark for Federated Machine Learning
• A Secure Federated Learning Framework for 5G Networks
• LEAF: A Benchmark for Federated Settings
• Federated Learning with Bayesian Differential Privacy
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Split learning for health: Distributed deep learning without sharing raw patient data
• Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Federated Learning: Opportunities and Challenges
• Federated Forest
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
• Edge Intelligence: Architectures, Challenges, and Applications
• GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning
• Abnormal Client Behavior Detection in Federated Learning
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysis
• Privacy in Deep Learning: A Survey
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
• Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
• Federated Learning with Differential Privacy: Algorithms and Performance Analysis
• Evaluating Differentially Private Machine Learning in Practice
• Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Differential Privacy Has Disparate Impact on Model Accuracy
• Free-riders in Federated Learning: Attacks and Defenses
• Federated Learning for Healthcare Informatics
• A Systematic Survey of Blockchained Federated Learning
• Differentially Private Learning Needs Better Features (or Much More Data)
• Local Differential Privacy and Its Applications: A Comprehensive Survey
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
• Label Leakage and Protection in Two-party Split Learning
• Robust Federated Learning: The Case of Affine Distribution Shifts
• Private Federated Learning with Domain Adaptation
• Attack-Resistant Federated Learning with Residual-based Reweighting
• Communication-Computation Efficient Secure Aggregation for Federated Learning
• Oort: Efficient Federated Learning via Guided Participant Selection
• FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
• Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
• Differentially-Private Federated Linear Bandits
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
• Eavesdrop the Composition Proportion of Training Labels in Federated Learning
• Federated Principal Component Analysis
• LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
• Asynchronous Federated Learning with Differential Privacy for Edge Intelligence
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• Federated Model Distillation with Noise-Free Differential Privacy
• Wireless Federated Learning with Local Differential Privacy
• FEDZIP: A Compression Framework for Communication-Efficient Federated Learning
• Federated Reconstruction: Partially Local Federated Learning
• A Federated Learning Approach for Mobile Packet Classification
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Enhancing the Privacy of Federated Learning with Sketching
• GFL: A Decentralized Federated Learning Framework Based On Blockchain
• MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
• Differentially Private Meta-Learning
• D2P-Fed: Differentially Private Federated Learning With Efficient Communication
• Helios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration
• SPEED: Secure, PrivatE, and Efficient Deep learning
• On Safeguarding Privacy and Security in the Framework of Federated Learning
• Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
• An Overview of Privacy in Machine Learning
• COVID-19 Imaging Data Privacy by Federated Learning Design: A Theoretical Framework
• Fast-adapting and Privacy-preserving Federated Recommender System
• Auditing Data Provenance in Text-Generation Models
• A Theoretical Perspective on Differentially Private Federated Multi-task Learning
• FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
• Deep Learning for Biomedical Image Reconstruction: A Survey
• FedCon: A Contrastive Framework for Federated Semi-Supervised Learning
• Federated Heavy Hitters Discovery with Differential Privacy
• Combining Federated and Active Learning for Communication-efficient Distributed Failure Prediction in Aeronautics
• Quantification of the Leakage in Federated Learning
• Turn Signal Prediction: A Federated Learning Case Study
• User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization
• Optimal query complexity for private sequential learning against eavesdropping
• Gradient Descent with Compressed Iterates
• Layer-wise Characterization of Latent Information Leakage in Federated Learning
• Sustainable Federated Learning
• FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
• Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges
• From Federated Learning to Federated Neural Architecture Search: A Survey
• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture
• FedSiam: Towards Adaptive Federated Semi-Supervised Learning
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times
• Byzantine-Resilient Secure Federated Learning
• Federated Learning System without Model Sharing through Integration of Dimensional Reduced Data Representations
• BlockFLow: An Accountable and Privacy-Preserving Solution for Federated Learning
• Weighted Distributed Differential Privacy ERM: Convex and Non-convex
• Federated Intrusion Detection for IoT with Heterogeneous Cohort Privacy
• Federated $f$-Differential Privacy
• Differentially Private Federated Learning with Laplacian Smoothing
• Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints
• Anonymizing Data for Privacy-Preserving Federated Learning
• Stochastic Channel-Based Federated Learning for Medical Data Privacy Preserving
• DistFL: Distribution-aware Federated Learning for Mobile Scenarios
• The More, the Better? A Study on Collaborative Machine Learning for DGA Detection
• Federated Learning on Non-IID Data: A Survey
• PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy
• On the Convergence and Calibration of Deep Learning with Differential Privacy
• Concentrated Differentially Private and Utility Preserving Federated Learning
• Privacy-preserving Federated Bayesian Learning of a Generative Model for Imbalanced Classification of Clinical Data
• Practical One-Shot Federated Learning for Cross-Silo Setting
• Towards Fair Federated Learning with Zero-Shot Data Augmentation
• SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation
• Benchmarking Differential Privacy and Federated Learning for BERT Models
• Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning
• Security of Deep Learning Methodologies: Challenges and Opportunities
• Efficient Privacy Preserving Edge Computing Framework for Image Classification
• Voting-based Approaches For Differentially Private Federated Learning
• ESMFL: Efficient and Secure Models for Federated Learning
• OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithms
• Federated Multi-Armed Bandits
• Confederated Machine Learning on Horizontally and Vertically Separated Medical Data for Large-Scale Health System Intelligence
• Federated Learning with Positive and Unlabeled Data
• Privacy-preserving Transfer Learning via Secure Maximum Mean Discrepancy
• Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
• Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
• Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent
• Federated Learning in Adversarial Settings
• Addressing Class Imbalance in Federated Learning
• Understanding the Tradeoffs in Client-side Privacy for Downstream Speech Tasks
• PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization
• Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communication
• WAFFLE: Watermarking in Federated Learning
• Source Inference Attacks in Federated Learning
• Trends and Advancements in Deep Neural Network Communication
• Policy-Based Federated Learning
• Decentralized Differentially Private Segmentation with PATE
• Private Multi-Task Learning: Formulation and Applications to Federated Learning
• An Exploratory Analysis on Users' Contributions in Federated Learning
• YourAdvalue: Measuring Advertising Price Dynamics without Bankrupting User Privacy
• Robust Federated Learning with Attack-Adaptive Aggregation
• P3SGD: Patient Privacy Preserving SGD for Regularizing Deep CNNs in Pathological Image Classification
• Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning
• Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications
• Privacy-Preserving Collaborative Deep Learning with Unreliable Participants
• An End-to-End Encrypted Neural Network for Gradient Updates Transmission in Federated Learning
• High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions
• Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
• Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection
• Critical Learning Periods in Federated Learning
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX
• Privacy-Preserving Federated Learning on Partitioned Attributes
• Towards Causal Federated Learning For Enhanced Robustness and Privacy
• Learn distributed GAN with Temporary Discriminators
• An Empirical Study on the Intrinsic Privacy of SGD
• Federated Extra-Trees with Privacy Preserving
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• Separation of Powers in Federated Learning
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices
• Dynamic Attention-based Communication-Efficient Federated Learning
• Preliminary Steps Towards Federated Sentiment Classification
• Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses
• GRAFFL: Gradient-free Federated Learning of a Bayesian Generative Model
• Knowledge Transferring via Model Aggregation for Online Social Care
• FDNAS: Improving Data Privacy and Model Diversity in AutoML
• Domain Adaptation without Model Transferring
• Fair and autonomous sharing of federate learning models in mobile Internet of Things
• Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy
• Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning
• Federated Unbiased Learning to Rank
• A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks
• Semi-Federated Learning
• Knowledge Federation: A Unified and Hierarchical Privacy-Preserving AI Framework
• Gradient-Leakage Resilient Federated Learning
• A Privacy-Preserving and Trustable Multi-agent Learning Framework
• Pronto: Federated Task Scheduling
• FedNNNN: Norm-Normalized Neural Network Aggregation for Fast and Accurate Federated Learning
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
• Decentralized Wireless Federated Learning with Differential Privacy
• Topology-aware Differential Privacy for Decentralized Image Classification
• Preventing Adversarial Use of Datasets through Fair Core-Set Construction
• Federated Learning from Small Datasets
• Federated Marginal Personalization for ASR Rescoring