Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning

Abstract

Machine learning (ML) has progressed rapidly during the past decade and the major factor that drives such development is the unprecedented large-scale data. As data generation is a continuous process, this leads to ML model owners updating their models frequently with newly-collected data in an online learning scenario. In consequence, if an ML model is queried with the same set of data samples at two different points in time, it will provide different results. In this paper, we investigate whether the change in the output of a black-box ML model before and after being updated can leak information of the dataset used to perform the update, namely the updating set. This constitutes a new attack surface against black-box ML models and such information leakage may compromise the intellectual property and data privacy of the ML model owner. We propose four attacks following an encoder-decoder formulation, which allows inferring diverse information of the updating set. Our new attacks are facilitated by state-of-the-art deep learning techniques. In particular, we propose a hybrid generative model (CBM-GAN) that is based on generative adversarial networks (GANs) but includes a reconstructive loss that allows reconstructing accurate samples. Our experiments show that the proposed attacks achieve strong performance.

USENIX Security 2020

Cited by in corpus (25)

• BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
• Analyzing Information Leakage of Updates to Natural Language Models
• Systematic Evaluation of Privacy Risks of Machine Learning Models
• Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities
• Defending Model Inversion and Membership Inference Attacks via Prediction Purification
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
• Stealing Links from Graph Neural Networks
• Eavesdrop the Composition Proportion of Training Labels in Federated Learning
• Leakage of Dataset Properties in Multi-Party Machine Learning
• Bilateral Dependency Optimization: Defending Against Model-inversion Attacks
• Towards General Deep Leakage in Federated Learning
• Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture
• BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models
• Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networks
• Personalized Graph Federated Learning with Differential Privacy
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• Towards Security Threats of Deep Learning Systems: A Survey
• Introducing Model Inversion Attacks on Automatic Speaker Recognition
• FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries
• Quantifying and Mitigating Privacy Risks of Contrastive Learning
• Addressing Class Imbalance in Federated Learning
• Teacher Model Fingerprinting Attacks Against Transfer Learning
• DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing
• Privacy-Preserving Image Acquisition Using Trainable Optical Kernel