Sharing Models or Coresets: A Study based on Membership Inference Attack

Abstract

Distributed machine learning generally aims at training a global model based on distributed data without collecting all the data to a centralized location, where two different approaches have been proposed: collecting and aggregating local models (federated learning) and collecting and training over representative data summaries (coreset). While each approach preserves data privacy to some extent thanks to not sharing the raw data, the exact extent of protection is unclear under sophisticated attacks that try to infer the raw data from the shared information. We present the first comparison between the two approaches in terms of target model accuracy, communication cost, and data privacy, where the last is measured by the accuracy of a state-of-the-art attack strategy called the membership inference attack. Our experiments quantify the accuracy-privacy-cost tradeoff of each approach, and reveal a nontrivial comparison that can be used to guide the design of model training processes.

References in corpus (6)

• Federated Learning: Strategies for Improving Communication Efficiency
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
• Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach

Cited by in corpus (1)

• Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI